1 EVALUATION Performance Evaluation of the Socio-Economic Development Activity (SEDA) January 2016 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Mendez England & Associates. 2 USAID/AZERBAIJAN PERFORMANCE EVALUATION OF THE SOCIO ECONOMIC ACTIVITY (SEDA) Final Report January 2016 Prepared under RFTOP No. SOL-112-15-000002 Submitted to: USAID/Azerbaijan Submitted by: Dr. James M. Statman, Team Leader Zehra Kacapor-Dzihic, Evaluation Specialist/Technical Advisor Bahruz Babayev, Local Consultant Contractor: Mendez England & Associates 4300 Montgomery Avenue, Suite 103 Bethesda, MD 20814 Tel: 301- 652 -4334 www.mendezengland.com DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. 3 CONTENTS EXECUTIVE SUMMARY.....................................................................................Error! Bookmark not defined. 1.0 EVALUATION PURPOSE AND QUESTIONS ............................................................................................ 7 1.1 Evaluation Background.................................................................................................................... 7 1.2 Evaluation Purpose....................................................................................................................... 7 1.3 Evaluation Questions...................................................................................................................... 7 2.0 PROJECT BACKGROUND............................................................................................................................. 8 2.1 Background...................................................................................................................................... 8 2.2 Development Context.................................................................................................................... 8 3.0 EVALUATION METHODS AND LIMITATIONS............................................................................................ 9 3.1 Evaluation Methodology ................................................................................................................. 9 3.1.1 Qualitative Research Analysis................................................................................................... 9 3.1.2 Quantitative Research Analysis............................................................................................... 10 3.1.3 Gender Considerations............................................................................................................ 11 3.2 Evaluation Limitations.................................................................................................................. 11 4.0 FINDINGS ............................................................................................................................................................ 2 4.1 Evaluation Question One: How effective has been the implementation of each SEDA component?.................................................................................................................................. 11 4.1.1 Component 1: Revitalization or establishment of village and/or district-based CDCs to increase community participation in socio-economic development at the local and regional levels..................................................................................................... 11 4.1.2 Component 2: Strengthening the capacity of CSOs to engage with citizens and government officials on local, regional, and national socio-economic priorities. ....... 15 4.1.3 Component 3: Strengthening the capacity of the (GoAJ) to engage with stakeholders on key socio-economic development issues............................................... 16 4.2 Evaluation Question Two: How complimentary were the activities under the three components identified in the scope of the project towards each other? Which interventions were the most effective towards achieving complementarity of results?........ 17 4.3 Evaluation QuestionThree: How sustainable are the established dialogue and relationships between CDCs, other civil society and government officials?................................................. 18 4.4 Evaluation Question Four: How successful was SEDA project in integrating gender considerations.............................................................................................................................. 19 4 5.0 CONCLUSIONS .............................................................................................................................................. 20 6.0 RECOMMENDATIONS.................................................................................................................................. 21 7.0 LESSONS LEARNED ....................................................................................................................................... 22 ANNEXES: ANNEX 1: SOW ANNEX 2: Work Plan and Design ANNEX 3: Evaluation Instruments ANNEX 4: Evaluation Schedule ANNEX 5: List of Interviewees ANNEX 6: Site Visits Meetings and Interviews ANNEX 7: CDC Profile ANNEX 8: Survey of CDC and CSO Members ANNEX 9: Surveys with CSO Participating in SPPU Training ANNEX 10: Data from CDCs ANNEX 11: SPPU Survey Data ANNEX 12: Documents Reviewed 1 ACRONYMS AZRIP Azerbaijan Rural Investment Project CSO Civil Society Organization CDC Community Development Council CEG Civic Engagement Grant CSSN (or NGO Council) Council for State Support to Non-governmental Organizations under the President of the Republic of Azerbaijan EWMI East-West Management Institute ExCom Executive Committee ET Evaluation Team ERC Economic Research Center FGD Focus group discussion GoAJ Government of Azerbaijan Republic ICNL International Center for Not-for-Profit Law MEI Ministry of Economy and Industry NNF National NGO Forum PPP Public Private Partnerships PRD Platforms for Regional Dialogue PC Public Council PMP Performance Monitoring Plan Rayon Administrative district RNRTC Regional NGO Resource and Training Center SEDA Socio-Economic Development Activity SOW Statement of Work SPPU Social Progress Public Union UMID Support to Social Development Public Union 2 EXECUTIVE SUMMARY EVALUATION PURPOSE This is a report on the Performance Evaluation of the Socio Economic Activity (SEDA) project funded by the United States Agency for International Development (USAID) Mission in Azerbaijan. SEDA is being implemented by the East-West Management Institute (EWMI) and several local partners, including: Support to Social Development Public Union (UMID); the Azerbaijan National Non-Governmental Organizations (NGO) Forum and Regional NGO Resource and Training Centers; and the International Center for Not-for-Profit Law (ICNL)/MG Consulting. SEDA is co-financed by USAID/Azerbaijan and the Government of Azerbaijan (GoAJ), and has a total estimated cost of approximately $14,399,500. The evaluation of SEDA was conducted during the period November – December 2015 by a team assembled by Mendez England & Associates (ME&A) located in Bethesda, Maryland. The purpose of the evaluation is to determine the effectiveness and complementarity of SEDA’s three core program components during the four year period from 30 September 2011 to 30 September 2015, and to provide recommendations to improve project implementation and results for the remaining period. PROJECT BACKGROUND SEDA is a five-year initiative that seeks to enhance civic participation in local and regional governance and strengthen the capacity of civil society and local governing authorities to productively consult and collaborate on identifying and addressing local socio-economic development priorities. The project is organized around three interrelated core programmatic components. • Component 1 focuses on local socio-economic development and community engagement in small, often-marginalized communities in Aran, Ganja and Guba-Khachmaz regions. The purpose is to: 1) Revitalize and/or establish local Community Development Councils (CDCs); 2) Strengthen the CDCs’ capacity through training, networking, information and experience sharing; and 3) Assist the communities to identify, prioritize, plan and implement small, socio￾economic infrastructure development projects. • Component 2 supports and strengthens civil society organizations (CSOs) in the SEDA target regions. SEDA provides capacity-building training and information dissemination to CSOs and Regional NGO Resource and Training Centers. • Component 3 works to strengthen the capacity of the GoAJ to engage with stakeholders on key socio-economic development issues. At the time of this performance evaluation, SEDA reported to have worked in 96 communities across the three target regions. EVALUATION QUESTIONS As specified in the SEDA Performance Evaluation Statement of Work (SOW) the evaluation has been organized to specifically address four evaluation questions: 1. How effectively has each component been implemented? 2. How complementary are the activities addressing each component and which interventions were the most effective towards achieving complementarity of results? 3. How sustainable are the dialogue and relationships established between CDCs, and other civil society and government officials? and 3 4. How successful is SEDA in integrating gender considerations? EVALUATION DESIGN AND METHODS The evaluation methodology employed an array of complementary assessment techniques, including: quantitative and qualitative data collection; individual and group responses; focus group discussions (FGDs) and surveys; direct observations; and community discussions, among others. Primary data collection for the performance evaluation was conducted in Azerbaijan over the course of approximately 17 days in December 2015. Data was collected through the following methods: • Document Review: The Evaluation Team (ET) reviewed a comprehensive set of documents related to SEDA and the context in which it was implemented provided by USAID, the EWMI and other local informants. • Key Informant Interviews (KIIs): The ET interviewed 20 persons with knowledge of SEDA’s objectives, activities, beneficiaries and/or outcomes. Interviewees included USAID officials in Azerbaijan; SEDA team members in Baku and in the targeted regions; CSO representatives; experts in civil society issues in Azerbaijan; and local government officials. • Surveys: The ET conducted three surveys: two with CDCs and community members and one with CSOs that were visited in the field. In total, the ET received responses from 122 participants (86 from CDC and community members; 16 from CSOs visited in the field; and 20 from all CSOs receiving capacity building support from the Regional Training Center in Qabala). All surveys were entered and administered through Survey Monkey™. • Focus Group Discussions (FGDs): The ET conducted 14 FGDs with representatives of CDCs that were formed or supported by SEDA and which received small grants for infrastructure projects. 98 people participated in the FGDs, including 67 females. The ET also conducted group discussions with CSOs and Resource centers in two regions involving 13 CSOs with 16 participants (seven males and nine females). • Site Observations: During the field visits to SEDA target communities, the ET observed a total of 13 small infrastructure projects EVALUATION LIMITATIONS The primary limitation relates to possible sampling bias. Due to time constraints, the ET could not conduct random sampling of project sites and beneficiaries. Therefore, targeted purposive and opportunistic sampling was applied to generate data to reliably address the four key evaluation questions. Although the data collected and analyzed presents a plausible and coherent account, the lack of random sampling means that these findings cannot be strictly generalized to the population as a whole. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS Evaluation Question One: How effective has been the implementation of each SEDA component? • Site visits and KIIs revealed that SEDA provided community members with concrete skills in project design and implementation, increased hope in a positive future for their village, and a sense that as a community they can work with government to address key community needs. • FGDs revealed frustration on the part of some CDC and community members as to what they perceive as long waits for final infrastructure project approval. • SEDA has delivered a number of capacity building activities for CSOs, which, according to surveys and FGDs, are judged to be quite successful by the participants. • SEDA deployed a multi-disciplinary group of leading Azerbaijani and international organizations 4 and experts to assist local and central government institutions to develop their capacity to engage citizens, CSOs, and the private sector in developing strategies and budgets for socio￾economic regional development. The project has played an important role in supporting, promoting and shaping the Law on Public Participation and building public awareness of the role of Public Councils. Evaluation Question Two: How complimentary were the activities under the three components identified in the scope of the project towards each other? Which interventions were the most effective towards achieving complementarity of results? • SEDA’s design of three complementary, integrated components synergistically contributing to the achievement of overall program objectives is a conceptually sound model. However, due to exogenous political factors outside the boundaries of SEDA’s control that constrained operations of CSOs, the complementarity between all three components could not be achieved. • Evaluation data indicate that the training and public awareness interventions and results under Components 1 and 3 have demonstrated some complementarity. With the enactment of the Law on Public Participation, Public Councils at all levels of governance are expected to be established. At present, however, few Public Councils appear operational and there is no timetable in the Law mandating the actual establishment of these bodies. • The ET could not detect a significant NGO footprint in any of the SEDA villages it visited, nor find evidence that the Component 2 capacity-building training interventions have complemented or provided synergy with Component 3 activities. Evaluation Question 3: How sustainable are the established dialogue and relationships between CDCs, other civil society and government officials? • SEDA’s approach to strengthening the capacities of CDCs has created participatory structures and procedures that appear sustainable. Site visits and meetings with CDCs indicated that all visited continue to operate even after completion of SEDA’s infrastructure projects. Almost 20% of all CDCs have succeeded in securing an additional funding for a second or even third SEDA-sponsored project. Several CDCs have obtained other grants from international donors and large infrastructure improvements from GoAJ. • Municipal support for the on-going maintenance of SEDA-generated infrastructure development has been achieved in most villages visited by the ET, indicating the creation of strong linkages between local government and the voluntary CDC structures. • SEDA’s trainings, workshops, and other events have established and strengthened dialogue, networks and support between the various CDCs and GoAJ. Under Component 1, a strong relationship between the CDCs and the municipalities in 14 of 15 sites visited was described by informants. Component 2 and 3 training interventions often bring together GoAJ personnel with civil society counterparts but at this point it is too early to tell whether this dialogue will be sustained after project conclusion. Evaluation Question Four: How successful was SEDA project in integrating gender considerations? • While SEDA has worked diligently to integrate gender awareness into all of its activities, it has not consistently achieved gender equality in terms of actual participation in these activities. Analysis of PMP data shows that out of a total of 11,272 people participating in trainings within the three components, only 37% are female. CDCs, however, have a good gender representation, with over half having 41-50% female memberships and 12% having more than 50% women members. 5 CONCLUSIONS 1. Overall, SEDA has been conscientiously and effectively implemented within the constraints of the political context in which it has operated. • Component 1 has been effectively implemented: a) CDCs were established in all targeted communities; b) CDCs generated increased community participation level; and c) local SEDA￾supported infrastructure projects have had a modest effect on improving the socio-economic situation in the communities. Yet, they were effective in providing community members with new skills and a sense of solidarity, empowerment, and hope in the future. • Component 2 could not be effectively implemented as designed. While CSOs may play a valuable future role in socio-economic development, within the present political context the effectiveness of a CSO-based strategy is limited. 2. Component 3 has been effectively implemented, within the limitations of the current political environment. SEDA’s contribution to the enactment of the Law on Public Participation is significant. Although conceptually, complementarity and synergies between the three SEDA program components would be expected, within the complex and fraught political environment in which SEDA is actually being implemented, the potential for complementarity particularly for Component 2, is constrained. • Some complementarity was demonstrated between SEDA’s Component 1 and 3. If the Public Councils mandated by the Law on Public Participation are established at the municipal level, the ET expects CDC members to play an active role and SEDA to be well-positioned to deliver capacity building support to the Councils at all levels. • Within the current context, CSO development (Component 2) cannot be effectively integrated within the SEDA model and does not add complementarity to the other program components. 3. Within the immediate future, SEDA’s results and relationships are expected to be largely sustained. • Many CDCs can be expected to remain functional into the near future. • CDCs are expected to maintain close working relationships with municipalities after SEDA ends. • The small, local and cluster infrastructure projects will likely be sustained by local authorities, CDCs, and community members. • The skills and knowledge acquired through SEDA training and the experience of community mobilization and participatory development will be retained and can be applied in addressing local development outside of the SEDA framework. 4. Challenges remain in achieving full gender equity in SEDA program activities. • SEDA has made a diligent effort to bring gender equity to all of its activities and insisted on women’s participation in all activities including CDCs. Still, the participation of women has not been fully equitable across all of the SEDA activities. RECOMMENDATIONS 1. SEDA should continue to engage and provide support to the GoAJ within the purview of its SOW and strengthen linkages and complementarity of these activities with its local socio-economic development initiatives. 2. Cluster projects may offer an opportunity for mobilized communities to cooperatively engage in addressing larger socio-economic priorities. As such, they should be further explored by SEDA. 3. SEDA/USAID should consider the potential benefits and trade-offs of increasing the size of SEDA’s infrastructure grants. 6 4. Resources permitting, the current Component 1 model of CDC development could be productively expanded to additional regions of Azerbaijan. 5. Should in the present economic context, USAID/Azerbaijan decide to focus on distinct economic development objectives, traditional program models of entrepreneurship training and support, SME and value-chain development must be considered in conjunction with the MoEI. 6. Assuming the creation of Public Councils in the near future: - Assistance should be provided to CDC members in transitioning to roles as members of local Public Councils. - Training support should be offered to Public Councils and assistance provided to the GoAJ in establishing monitoring and accountability systems for Councils at all levels. 7. If establishment of Public Councils is delayed, further assistance to CDCs is advisable to support their sustainability and maintain community morale. 8. At this stage of SEDA’s implementation, if there is improvement in the enabling environment, a CSO-based community development strategy should be created as a distinct program rather than be incorporated into SEDA. 9. Gender equity should be further emphasized in all subsequent activities, including training events, conferences and study tours. 10. Given the current momentum and upturn in program activities, if SEDA funds remain after the fifth and final year of the project, a no-cost extension should be considered to maximize program results. LESSONS LEARNED 1. SEDA’s success in facilitating the establishment and capacity-building of CDCs in more than 100 small and often remote and impoverished villages, and the communities’ enthusiasm and competence in planning, supporting and completing small but often strategic infrastructure development projects, validates its vision of local collaboration and participation. 2. The development context may prove contradictory, presenting unlikely opportunities as well as constraints. While a complex and fraught political environment may limit effective pursuit of some program objectives and activities - as was the case for CSO-development under Component 2 – space was open for the achievement of other related activities, particularly SEDA’s support for the enactment of the Law on Public Participation. 3. SEDA’s design, which combines three interrelated components, provides a kind of robustness and resiliency enabling the program to adapt and shift areas of emphasis, as necessary. 4. Despite a discouraging and, at times, threatening context for program implementation, rather than close down, SEDA/EWMI and USAID/Azerbaijan demonstrated that a commitment to development, adaptability, and flexibility, based on relationship-building and local knowledge can enable a program to successfully reconfigure and focus on areas of opportunity while maintaining its value and integrity. 7 1.0 EVALUATION BACKGROUND, PURPOSE AND QUESTIONS 1.1 EVALUATION BACKGROUND This is a report on the Performance Evaluation of the Socio Economic Development Activity (SEDA) project funded by the United States Agency for International Development (USAID) Mission in Azerbaijan, prepared under RFTOP No. SOL-112-15-000002. SEDA is being implemented by the East￾West Management Institute (EWMI) and several local partners, including: Support to Social Development Public Union (UMID); the Azerbaijan National Non-Governmental Organizations (NGO) Forum and Regional NGO Resource and Training Centers; and the International Center for Not-for￾Profit Law (ICNL)/MG Consulting. SEDA is co-financed by USAID/Azerbaijan and the Government of Azerbaijan (GoAJ), and has a total estimated cost of approximately $14,399,500. The evaluation of SEDA was conducted during the period November – December 2015 by a team assembled by Mendez England & Associates (ME&A) located in Bethesda, Maryland. The Evaluation Team (ET) comprised of three key experts: Dr. James Statman (Team Leader), Ms. Zehra Kacapor-Dzihic (Evaluation Specialist/Technical Advisor), and Mr. Bahruz Babayev, Local Consultant. In addition, the ET was supported by two local interpreters and logistics specialists. The findings, conclusions and recommendations in this report emanate from the collective efforts of the above-mentioned team. 1.2 EVALUATION PURPOSE The purpose of this Performance Evaluation was to determine the effectiveness and complementarity of SEDA’s three core program components during the four year period from September 2011 to September 2015, and to provide recommendations to improve project implementation and results. The results of the evaluation are intended to be used by USAID/Azerbaijan to consider possible changes to SEDA’s design or scope of work (SOW) during its final months of operation, as well as to inform possible follow-on program activities. Other potential audiences include USAID/Washington (Bureau for Europe and Eurasia Bureau for Democracy, Conflict and Humanitarian Assistance; and Bureau for Policy, Planning and Learning), EWMI, GoAJ (Ministry of Economy, Council for State Support to NGOs), and other USAID implementing partners and beneficiaries of the SEDA project activities, including civil society organizations (CSOs) and local communities. 1.3 EVALUATION QUESTIONS As specified in the SOW, the evaluation was organized to specifically address four evaluation questions: 1. How effectively has each component been implemented? 2. How complementary are the activities addressing each component and which interventions were the most effective towards achieving complementarity of results? 3. How sustainable are the dialogue and relationships established between community development councils (CDCs), and other civil society and government officials? and 4. How successful is SEDA in integrating gender considerations? The ET’s response to these questions is elaborated in the main body of the report in the following sections and is supported, where appropriate, by tables, figures and charts. 8 2.0 PROJECT BACKGROUND 2.1 BACKGROUND Commencing in 2011, SEDA is a five-year initiative that seeks to enhance civic participation in local and regional governance, as well as strengthen the capacity of civil society and local governing authorities to productively consult and collaborate on identifying and addressing local socio-economic development priorities. SEDA is being implemented by EWMI and local partners and in cooperation with GoAJ institutions at the central and local levels. The project is organized around three interrelated core programmatic components. • Component 1 focuses on local socio-economic development and community engagement in small, often marginalized communities in the Aran, Ganja and Guba-Khachmaz regions, and aims to: 1) Revitalize and/or establish local Community Development Councils (CDCs); 2) Strengthen the CDCs’ capacity through training, networking, information and experience sharing; and 3) Assist the communities to identify, prioritize, plan, and implement small, socio-economic, infrastructure development projects. • Component 2 aims to support and strengthen CSOs in SEDA’s target regions. Working with Regional NGO Resource and Training Centers, SEDA provides capacity-building training, information dissemination, and mentoring to these registered CSOs. • Component 3 works to strengthen the capacity of the GoAJ to engage with stakeholders on key socio-economic development issues through support of mechanisms and training for enhanced public participation in socio-economic decision making. 2.2 DEVELOPMENT CONTEXT The economy of Azerbaijan has enjoyed an influx of petrodollars triggered by the opening of the Baku￾Tbilisi-Ceyhan oil pipeline in 2005. Oil and gas revenues resulted in a strong increase in salaries and high growth rate in urban areas. Furthermore, Azerbaijan concluded an agreement to transport natural gas from the Shah Deniz II field to Western Europe in 2017, which is expected to provide another significant economic boost. Despite these positive developments, the World Bank forecasts that oil revenues will shrink over the coming years, and that revenues from natural gas will not be sufficient to compensate for this decline1. A critical challenge facing the GoAJ is the need to diversify the economy and determine a mechanism to effectively develop non-oil sectors. Socio-economic development of the regions of Azerbaijan remains an important interrelated priority for the Azerbaijani government2, with the agricultural and industrial sectors being critical3. During the oil boom, the Azerbaijani government invested significantly in development of Baku as a center for global cultural and sport events, which generated a boom in the construction sector. However, other regions of the country were relatively neglected; therefore, they presently suffer from inadequate economic development, limited employment opportunities, and poor infrastructure. The rural areas, in particular, often lack access to potable water, health and education facilities, viable road networks or bridges, and reliable electric power. These areas are characterized by rates of 1 http://www.worldbank.org/en/country/azerbaijan/overview 2 http://senaye.gov.az/content/html/2280/attachments/State%20Program%20on%20socio￾economic%20development%20of%20regions%20of%20the%20Republic%20of%20Azerbaijan%20for%202014-2018%20years.pdf 3 http://files.preslib.az/projects/repulic/en/azr3_6.pdf 9 unemployment, especially among youth, that are much higher than those in urban centers4. They also have large numbers of internally displaced persons (IDPs), an on-going legacy of the still unresolved and costly Karabagh war with Armenia. These IDPs also suffer from poor access to water, housing, and sanitation, and face high levels of unemployment and poverty5. The precipitous fall of international crude oil prices since mid-2014 has generated a major crisis in Azerbaijan’s economy, resulting in the cancellation of important investment projects and significant closure of small and medium sized businesses, thereby increasing unemployment rates. The local currency has significantly depreciated, resulting in steep price increase for many goods and services and a decrease in trade. Pushed by economic crisis, GoAJ has recently expressed a willingness to start reforms to liberalize customs and create a better environment for small and medium sized businesses. In addition, Azerbaijan has a complex political environment that, at times, has appeared hostile to the growth and role of civil society, and highly suspicious of foreign-donor supported development activities. While the situation may be somewhat abating, legislation regulating CSOs’ operations through restrictive organizational and grants registration processes and crack-downs by security agencies hase narrowed the space for the free operations of NGOs. All of the above provide an uncertain and challenging context for social and economic development of the country. It is within this difficult context that SEDA is operating and implementing its activities. 3.0 EVALUATION METHODS AND LIMITATIONS 3.1 EVALUATION METHODOLOGY The evaluation methodology was designed to specifically address the four main evaluation questions. In accordance with the approved SEDA Performance Evaluation Work Plan and Evaluation Design (see Annex 2), the methodology employed an array of complementary assessment techniques, including: quantitative and qualitative data collection; individual and group responses; focus group discussions (FGDs); surveys; direct observations; and community discussions, among others. These various overlapping methods enabled the ET to capture the diversity of opinions and perceptions of beneficiaries and stakeholders about the achievements, effectiveness, and sustainability of SEDA. Information from these methods was triangulated to test the reliability of the results. Any discrepant observations and data have been noted. Primary data collection for the performance evaluation was conducted in Azerbaijan over the course of approximately 17 business days starting in December 2015. 3.1.1 Qualitative Research Analysis Document Review: The ET reviewed a comprehensive set of more than 35 documents provided by USAID, EWMI, and other local informants that related to SEDA and the context in which it was implemented. These documents included, among others, the SEDA Agreement and modifications, 15 SEDA Quarterly Reports, Annual Reports, Performance Monitoring Plan (PMP) design and reports, training lists, and post-training assessment reports. During the course of the evaluation, the ET requested and quickly received from EWMI a number of additional documents, data, and details on program implementation and indicators related to SEDA’s local projects, beneficiaries and mechanisms. Additional documents were identified and sourced from 4 http://www.adb.org/sites/default/files/linked-documents/cps-aze-2014-2018-pa.pdf 5 http://www.adb.org/sites/default/files/linked-documents/cps-aze-2014-2018-pa.pdf 10 key informants and through independent research by the ET, primarily via the Internet. A complete list of the documents reviewed for the evaluation can be found in Annex 12. Key Informant Interviews (KIIs): Using semi-structured interviews, the ET interviewed a total of 20 persons with knowledge of SEDA’s objectives, activities, beneficiaries and/or outcomes. In addition to Baku, the ET conducted KIIs in Aran, and Quba-Khachmaz regions. The full list of interviewed informants is provided in Annex 5. Key informants interviewed included USAID officials; SEDA team members in Baku and in the regions; CSOs involved directly or indirectly in the program activities; representatives of GoAJ – the Ministry of Economy and Industry, and State Council for Support to NGOs; staff of other organizations; experts in civil society issues in Azerbaijan; and local government officials. The ET conducted interviews in English or Azeri, with simultaneous translation, based on questionnaires developed to suit the particularities of different categories of informants and respond to the core evaluation questions. Guides can be found in Annex 3. Focus Group Discussions (FGDs): The ET conducted 14 FGDs with representatives of CDCs that were formed or supported by SEDA and had received small grants for one or more small infrastructure projects. These FGDs aimed to gather more in-depth qualitative information on their experience with the project, their interaction and level of benefit from SEDA’s support, the utility and sustainability of their SEDA-acquired knowledge and skills, as well as the maintenance and sustainability of the new/rehabilitated community infrastructure projects. The CDC FGDs were conducted in villages in all three targeted regions: Sabirabad (Memishlar, Zanqana, and Ahmadabad); Imishli (Jashar); Baylagan (Ashaghi Chemenli); Samukh (Ali Bayramli); Goranboy (Seyidler, Qushchular, Xoylu); Quba (Pustaqasim, Alpan, and Rustov); Shabran (Amirkhanli and Surra); and Khachmaz (Aghayazi Buduq). All FGDs were conducted in the Azeri language with simultaneous translation. In total, 98 people participated, including 67 females. The questions used to guide these discussions are included in Annex 3. The ET also met with and conducted open-ended group discussions with CSO representatives and Regional NGO Resource and Training Center staff in two regions (Ganja Qazakh and Quba-Khachmaz), involving 13 CSOs with16 participants (seven males and nine females). These discussions focused on the overall situation and challenges facing NGOs in the region, and the role of the Resource Centers in building CSO capacity. Site Observations: During the field visits to SEDA target communities, the ET conducted site observations of the new/rehabilitated infrastructure projects supported by the program. The ET visited a total of 13 small infrastructure projects in local communities visited. 3.1.2 Quantitative Research Analysis Surveys: To enrich the quantitative data regarding the program, the ET carried out three surveys (two distributed by the ET during site visits for CDC and community members, and one for CSOs visited in the field), and a web-based survey for CSOs. The surveys supplemented the data collected by other means. A total of 86 questionnaires were collected from CDC and community members; 16 from CSOs visited in the field; and 20 from all CSOs receiving capacity building support from the Regional Training Center in Qabala. All surveys were entered and administered through Survey Monkey™ to ensure easily accessible, inexpensive, and rapid means of collecting and analyzing data. In addition, the ET conducted a quantitative analysis of the data provided by EWMI on characteristics of all 103 CDCs with which SEDA had worked, and generated a report presenting a profile of descriptive variables of these local structures, including Council composition and membership characteristics such as age, gender and occupation, and types and numbers of infrastructure projects. 11 3.1.3 Gender Considerations The ET took into consideration the political context and cultural realities in Azerbaijan, including gendered beliefs and practices affecting participation and response rates, particularly in more traditional and rural areas. The ET placed emphasis on ensuring that any limitations to women participation and articulation of experiences and feedback were mitigated by providing a safe and culturally-acceptable space for FGDs and ensuring appropriate gender representation. Participation of women and their ability to voice their feedback was highly satisfactory during this evaluation. For example, of the 98 participants of FGDs with CDCs, 67 were women. 3.2 EVALUATION LIMITATIONS The ET encountered the following limitations in data collection and analysis: 1. Due to time constraints, the ET could not conduct random sampling of project sites and beneficiaries. Therefore, targeted purposive and opportunistic sampling was applied to generate data to reliably address the four key evaluation questions. Although the data collected and analyzed presents a plausible and coherent account, the lack of random sampling means that these findings cannot be strictly generalized to the population as a whole (sampling bias). 2. Since a number of questions during the interviews dealt with issues that took place in the past, recall bias cannot be excluded. As SEDA’s activities were launched in 2011, some respondents found it difficult to accurately compare organizational arrangements/access to services three years from prior to now. 3. There is a general tendency among respondents to under-report socially undesirable answers and alter their responses to approximate what they perceive as the social norm (halo bias). The extent to which respondents revealed their true opinions may also vary for some questions that call upon the respondents to assess the performance of their colleagues or people on whom they depend upon for the provision of services. To mitigate this limitation, ME&A provided the respondents with confidentiality and anonymity guarantees, where possible; conducted the interviews in the settings where respondents felt comfortable; and established rapport between the interviewer and the respondent. There were however, instances in which representatives of local authorities participated in the discussions which may have influenced the willingness of participants to openly discuss sensitive issues. 4.0 FINDINGS 4.1 EVALUATION QUESTION ONE: HOW EFFECTIVE HAS BEEN THE IMPLEMENTATION OF EACH SEDA COMPONENT? 4.1.1 Component 1: Revitalization or establishment of village and/or district-based CDCs to increase community participation in socio-economic development at the local and regional levels. Component 1 supports enhanced citizen participation in governance and implementation of socio￾economic development initiatives at the local level in the three target regions through the following sets of strategies: 1. Establish and strengthen CDCs 2. Support community-driven socio-economic projects; and 3. Support cluster community-driven projects 12 SEDA targeted both ‘pre-mobilized’ communities6 and ‘new’ communities7 in the three target regions. Of all participating local communities, one-third (32.7%) were ‘pre-mobilized’ – meaning they had experienced community development activities prior to SEDA’s intervention. As depicted in the pie￾chart below, more than half of the CDCs (55%) were established in 2014 and 2015, reflecting the increase in SEDA’s activities in the past two years, while almost all of the rest were established in 2012 and 2013. There are five CDCs that were established even earlier (between 1995 and 2010). As Table 1 shows, the Aran region hosts the greatest number of CDCs (42 or 40.8% of the project total), followed by Guba (36 or 35%), and Ganja (25 or 24.3% of the total CDCs). The CDC profile analysis of these communities conducted by the ET showed that in 86.4% of these communities, no local/community NGO is active. Table 1: CDCs in targeted regions Region Response Percent Response Count Guba 35,0% 36 Ganja 24,3% 25 Aran 40,8% 42 answered question 103 SEDA applies a comprehensive approach towards building capacities of CDCs to prioritize and develop proposals for community (infrastructure) projects, while at the same time nurturing community participation. The project delivered a range of training opportunities for CDCs, mentoring and facilitation of prioritization process, as well as intensive assistance and support in the development and implementation of infrastructure projects. 6 Pre-mobilized communities are those where either CDCs were previously established or community-driven socio-economic development projects were implemented under the Community Development Activity (CDA) project or other donor programs, such as the World Bank/Azerbaijan Rural Investment Project (AZRIP) and BP-sponsored programs. 7 New communities are those with no previous experience with CDCs and community-driven socio-economic development projects. 26% 18% 20% 35% Chart 1: When was the Council Created? 2012 2013 2014 2015 13 Findings After initial delays of more than one year, mainly related to political challenges in establishing functioning operational relationships with relevant GoAJ partners, SEDA effectively moved forward to engage 103 local communities - exceeding its Component 1 target of working in up to 100 communities - to mobilize local citizens to identify, plan, and implement small socio-economic infrastructure development projects. Site observations and field visits to a random selection of 15 target communities (14.6% of the total) found that many of these villages are extremely remote and all are quite impoverished, indicating that SEDA has selected and targeted communities appropriate to its program design. In all of the villages visited and during the FGDs, CDC and community members described an identical process for community mobilization and infrastructure project development. During initial visits, a staff member from SEDA local implementing partner, Support to Social Development Public Union (UMID), would conduct a large open meeting with community members to present the SEDA process for local community mobilization and infrastructure project development, and describe the role and objectives of the local voluntary, non-formal CDCs to be established or revitalized under the SEDA project. A CDC election was then conducted, with SEDA stressing the requirement for significant participation of women in this voluntary structure. CDC members were then provided with SEDA training courses in areas such as project planning, needs assessment, and project management, and were given opportunities to visit other SEDA-mobilized villages and to meet other CDCs. FGDs showed uniform enthusiasm and support from CDC members for the quality and utility of the training and visits to other villages. Almost 81% of 85 CDC members completing the ET’s survey rated the courses as extremely effective, and 12.6% as effective. Once organized and trained, the CDCs conducted a community needs assessment prioritization process, in some instances through one or more large community discussions and voting, and in others through door-to-door visits to all community residents to obtain their input and vote on infrastructure development priorities. As potential projects were identified, the CDCs worked closely with SEDA’s staff to determine the viability of the project and its compliance (or not) with SEDA’s technical and cost parameters, and to prepare technical specifications. All such local infrastructure projects required a minimum of 25% local resource match, a buy-in mechanism generally provided through volunteer community labor in project construction, and through donations of materials and supplies by local municipalities. The ET’s secondary analysis of EWMI data on 1038 CDCs found that the majority of CDCs (73.8%) had developed and submitted to SEDA proposals for two or more infrastructure projects. 73.1% of the CDCs have been awarded a grant for a single infrastructure project; 15.4% received two grants; 4.8% three or more; and 6.7% have not yet received final approval for a proposed project. While 44% of projects were completed by the time of the evaluation, 56% of projects are not yet finished. The SEDA-approved projects address a variety of communal issues: 34% involve road construction, 31% health, 32% schools or pre-school, and 25% water. As presented in Chart 2 below, there are also projects in the areas of recreation, bridges, agriculture, sports, electricity, and business development. 8 EWMI/SEDA reports a total of 96 operational CDCs. However, 103 CDCs appear in the SEDA data that the ET used for preparing the CDC profile report. For purposes of data analysis of the total CD population the ET has used the figure of 103. 14 During the site visits to 15 SEDA-targeted villages, the ET observed a wide variety of completed and under development infrastructure projects, including: inner roadways which according to village residents significantly reduced the time it takes them to get to/from their fields and for children to get to school; a bridge that has greatly improved village access; health points that provide significantly improved access to primary care; school infrastructure improvements that also serve as community meeting places; a communal agricultural storage facility, and sports fields. All seemed well-constructed and maintained, and villagers expressed pride in them and in their role in the design and construction of these projects. During FGDs, this dimension of community efficacy was mentioned often, with CDC members commenting that the SEDA process gave them concrete skills in project design and implementation, increased hope in a positive future for their village, and a sense that as a community they can work with government to address key community needs. In this regard, approximately 18% of CDCs reported that they have received funding from other sources to engage in further community socio-economic development. The FGDs revealed two areas of criticism pertaining to Component 1 that were voiced by participants. Several CDCs reported frustration at long waits for final infrastructure project approval that appear to relate to: 1) a sometimes lengthy process for environmental approval from USAID; and 2) the necessity for the final grant award process to be conducted at the EWMI home office in New York, due to the inability of EWMI/SEDA to be formally registered in Azerbaijan. These delays were embarrassing and discouraging for CDC members who were publically criticized by some skeptical local community members that doubted the projects would ever actually transpire. The second issue relates to the funding limits inherent in SEDA’s design itself. Although participants acknowledged that they were clearly informed of and understood SEDA’s project goals, procedures and funding limitations, some nevertheless expressed regret that the infrastructure projects could not be of a greater magnitude. While the priority infrastructure projects completed under SEDA are expected to make a genuine difference, as small-scale projects their capacity to effectively address fundamental community social and economic issues is obviously limited. Major structural projects involving water, roads, and electrical infrastructure are beyond SEDA’s scope and require significant GoAJ inputs. At one village, for example, while a new footbridge to an otherwise isolated community is certainly helpful, a larger but far more costly bridge that could accommodate vehicles would have been far preferable to the community. And, as the local CDC Chairman explained, the actual primary need of the village is for potable drinking water. But this priority has been unfulfilled despite, he reports, several appeals to government and an on-going history of water related illnesses in a large number of village children. So, 19% 22.40% 2.59% 1.72% 18% 2.58% 8.60% 11.20% 7.70% 5% 0.80% 0% 5% 10% 15% 20% 25% Chart 2. What did he Approved Projects Cover? 15 while all participating communities expressed gratitude and enthusiasm for SEDA, some community members felt frustration at its limitations. Within the Component 1, cluster projects – initiatives where multiple communities join forces to solve common infrastructure challenges – have also been initiated thus far on a somewhat limited basis. The SEDA implementation strategy in this respect has been incremental, focusing on first building and strengthening individual CDCs and on completion of local community infrastructure projects prior to working on more complex cluster activities9. The complexities inherent in cluster projects, in which two or more communities and their CDCs need to reach a common agreement on project priorities and coordinate inputs and activities, make this a more time consuming process. Site visits to villages participating in cluster projects showed the benefits of enabling two or more communities to address a common priority infrastructure issue. The ET visited two communities, which participated in cluster projects (one, for building a regional electric network, and the second for improving milk collection and storage for several villages), and the feedback on this approach was extremely positive. There are also examples of clusters of communities preparing new proposals for other donors, which suggest the on￾going benefits and sustainability of such cluster efforts. 4.1.2 Component 2: Strengthening the capacity of CSOs to engage with citizens and government officials on local, regional, and national socio-economic priorities. SEDA aimed at strengthening the civic engagement of CSOs in the targeted regions through investments in: capacity building; networking between non-governmental, governmental, and private sector stakeholders that push for community-driven regional socio-economic development; and offering opportunities to implement community projects through grants. Before having to terminate this activity due to restrictive changes in the NGO law and the general constraining environment regarding CSOs, SEDA was able to award six Civic Engagement Grants (CEGs) to local NGOs; however, these were discontinued. To maintain engagement with CSOs and deliver capacity-building training to them, following the “suggestion” from the CSSN that SEDA utilize an Azerbaijani NGO to implement activities under Component 2, in October 2014 SEDA competitively awarded a large grant to the Social Progress Public Union (SPPU), an established NGO with strong working ties to the NGO Council. SPPU conducted capacity building activities with CSOs and the Regional NGO Resource and Training Centers (RNRTCs), and provided direct support to the Centers in Guba and Shirvan. Findings SEDA’s investment in CSO strengthening was affected significantly by the contextual factors, primarily by the changes in the legislation governing the work and registration of NGOs and of grants from foreign donors. Document review and KIIs with USAID, EWMI/SEDA, and ICNL indicate that the revision of the NGO Law created unsurmountable obstacles for some NGO activities and, more importantly, narrowed the space for operation of foreign donors supporting civic activism and local initiatives. The obstacles to CSOs’ registration, the requirement that all grants from foreign donors be registered but lack of mechanism to do so, and the actions of state security agencies including raids on CSOs and the arrest of civil society activists have had a major impact in curtailing civil society activity. SEDA was also unable to award14 CEGs for “advocacy and watchdog activities” that had been reviewed and were “preselected” to be granted during project Years 2 and 3. The Director of a CSO that had been awarded a now-terminated CEG expressed great pessimism about possibilities for survival of 9 SEDA’s approach is that in clusters of two communities, both should have successfully completed a local infrastructure project prior to embarking on the cluster process. In instances where three communities participate in the cluster, at least two have completed local infrastructure projects. 16 independent NGOs working with international donors, and for the possibility of a truly independent civil society in Azerbaijan. While SEDA’s support for strengthening the space for an activist, changed-focused civil society was curtailed, its focus was shifted to training activities to build the institutional capacity of the less “political” service-focused NGOs that were registered within the legal framework of the new legislation. Desk reviews, interviews, and FGDs with CSO actors show that these activities, which focused on training delivery for CSO capacity building, were judged to be quite successful by the training participants. Capacity building and training delivered was focused on NGO organizational strengthening including, but not limited to, project planning and implementation trainings (74%), and advocacy or organizational capacity building trainings. The survey conducted among CSO participants in trainings, rated the trainings as very useful (73.7%) or useful (26.3%). Most beneficial for trainees was learning about decision-making, analysis, civil society, and gaining experience and knowledge. Also, respondents mentioned networking and assistance in project preparation and organization. In addition to NGO capacity-building training, during Project Years 3 and 4 under Component 2, SEDA organized two international study tours for CSO and CSSN personnel. The first, in September 2014, conducted in conjunction with the Bulgarian Center for Non-Profit Law, sent nine participants to Bulgaria to explore strategies and structures for CSO support and development. The second tour, which took the CSSN Chairman and six other CSSN and CSO representatives to Italy and was organized in conjunction with an Italian CSO, addressed issues of state support for CSOs. Survey respondents rated study tours either as “very useful” (83%), “useful” (8%) or of “average” use (9%). SEDA also launched Learning Circles – peer-to-peer dialog and support structures for CSO leaders to discuss pressing issues in each of its three focus regions, engaging 129 participants in six circles during Project Year 4. Learning circles were also highly rated as “very useful” or “useful” experiences by 93.3% of those completing the survey. Under Component 2, RNRTCs are both the recipients of SEDA’s capacity building assistance and providers of organization development, capacity-building training, and logistical support to the local NGOs that they serve. SEDA has helped RNRTCs conduct needs assessments and has provided an international expert to help them engage in a strategic planning process. CSO representatives from several local, regional, and national NGOs interviewed by the ET, reported that they have benefited from the training and support services provided by their RNRTC, which has strengthened their organizations’ operations and ability to apply for resources. 4.1.3 Component 3: Strengthening the capacity of the GoAJ to engage with stakeholders on key socio-economic development issues SEDA’s strategy towards strengthening the capacity of the GoAJ to engage with stakeholders in key economic development issues was multidimensional. The project deployed a multi-disciplinary group of leading Azerbaijani and international organizations and experts to assist local and central government institutions to develop their capacity to engage citizens, CSOs, and the private sector in developing strategies and budgets for socio-economic regional development. SEDA also initiated activities aimed at increasing governmental and private sector awareness of the potential benefits of Public Private Partnerships (PPPs) program. SEDA envisaged Component 3 to specifically target Executive Committees (ExComs), municipal, and regional directorates of central government institutions (such as the MoED Regional Economic Development Offices) in the SEDA-targeted regions, with priority given to municipalities and rayons with established CDCs and Platforms for Regional Development (PRDs), as well as central government institutions relevant to regional socio-economic development and CSO work. 17 The following activities were envisioned under this Component: - Capacity Development for Local and Central Government Officials - Enhancing Legislative Framework for Citizen Participation - Establishing Mechanisms for Citizen Participation; and - Piloting PPPs To implement this component, EWMI partnered with ICNL, a leading local organization with experience in working with local and central governments. In addition, the project engaged leading international experts on municipal development, public finance, participatory and gender budgeting, and PPPs. Findings Desk review and KIIs with representatives of the Council of State Support to Non-Governmental Organizations under the President of the Republic of Azerbaijan (CSSN), MG Consultants/ICNL10, the Constitution Research Fund, USAID, and EWMI show agreement on the positive role that SEDA has played in supporting, promoting, and shaping the Law on Public Participation. SEDA supported the development of the Law, including facilitation of regional public hearings on the draft law and as CSSN representatives noted, after the passage of the law worked effectively with the Council to conduct three regional roundtables on public participation. The CSSN also mentioned that SEDA also conducted a Monitoring and Evaluation training workshop for CSSN personnel to help build their capacity to monitor projects being implemented by its grantees. This activity is itself evidence of the positive working relationship between SEDA and this key GoAJ institution. As documented in the desk review, after conducting an initial training workshop examining and promoting Public Private Partnerships (PPPs) as a useful mechanism for socio-economic development, SEDA has found that the GoAJ has not opted to actively pursue such initiatives at present. KIIs with SEDA leadership found that SEDA continues to follow-up on this with Government partners and stands ready to move forward if and when the GoAJ wishes to do so. 4.2 EVALUATION QUESTION TWO: HOW COMPLEMENTARY WERE THE ACTIVITIES UNDER THE THREE COMPONENTS IDENTIFIED IN THE SCOPE OF THE PROJECT TOWARDS EACH OTHER? WHICH INTERVENTIONS WERE THE MOST EFFECTIVE TOWARDS ACHIEVING COMPLEMENTARITY OF RESULTS? SEDA’s design envisions three complementary, integrated components synergistically contributing to the achievement of overall program objectives. The project’s documents and KIIs with EWMI and USAID suggest that the project design incorporating three somewhat distinct components is a conceptually sound model for such complementarity. However, due to exogenous political factors outside the boundaries of SEDA’s control that constrained operations of CSOs, the complementarity between all three components was not achieved. Evaluation data indicate that the training and public awareness interventions and results under Components 1 and 3 have demonstrated some complementarity. With the enactment of the Law on Public Participation11 (Article 1.0.3), Public Councils at all levels of governance - including local municipalities – are expected to be established. Particularly at the municipal level, this result clearly 10 In light of legal restrictions on the registration and operations of international NGOs, MG Consultants, a Limited Liability Company, was established to continue the work of ICNL. 11http://www.commission-anticorruption.gov.az/upload/file/Law%20on%20Public%20Participation.pdf 18 builds upon and institutionalizes the CDC development mechanism, and reflects a complementarity of interventions. The establishment of Public Councils would also offer an appropriate and important opportunity for SEDA to provide capacity-building, technical support, and training to these new structures in furtherance of project objectives. At present, however, few, if any, Public Councils appear operational and there is no timetable in the Law mandating the actual establishment of these bodies. In light of the current severe economic crisis and its destabilizing impact upon the country’s economic, social, and political institutions it seems unlikely that there would be the political will to prioritize council development. At present, the ET could not detect a significant NGO footprint in any of the SEDA villages it visited. In addition, the successful participatory CDC mechanisms are attributable to the Component 1 community development activities linked to infrastructure development. The ET also could not find evidence that the Component 2 capacity-building training interventions have complemented or provided synergy with Component 3 activities. Faced with an altered political context leery of foreign donor support for civil society and severely constraining NGO registration and operations, Component 2 could not be effectively implemented, creating a gap in the SEDA model of mutual complementarity of the three components. The participatory intervention process through which CDCs are organized and the training that builds their capacity have potential complementarity with possible Public Councils and PPPs interventions under Component 3 and with possible CSO support for socio-economic development in SEDA targeted villages. At this point, there seems to be no attempt to link or integrate the CSOs supported by the project with the villages being mobilized. 4.3 EVALUATION QUESTION 3: HOW SUSTAINABLE ARE THE ESTABLISHED DIALOGUE AND RELATIONSHIPS BETWEEN CDCS, OTHER CIVIL SOCIETY AND GOVERNMENT OFFICIALS? SEDA’s design explicitly aims to sustain the relationships and dialogues it builds and reinforces between CDCs, GoAJ structures and officials, and CSOs. SEDA’s comprehensive approach to strengthening the capacities of CDCs and building transparent and inclusive prioritization of investments in local communities and for cluster projects has created participatory structures and procedures that appear to continue after the completion of the initial initiative. Site visits and meetings with CDCs indicate that all continue to operate even after completion of SEDA’s infrastructure projects. The ET’s CDC profiles analysis of all SEDA CDCs found that almost 20% have succeeded in securing an additional funding for a second or even third SEDA-sponsored project. Several CDCs have obtained other grants from international donors and large infrastructure improvements from GoAJ. Municipal support for the on-going maintenance of SEDA-generated infrastructure development has been accomplished in 10 of the 15 villages visited by the ET, indicating the creation of strong linkages between local government and the voluntary CDC structures. In the village of Ushcular, for example, the municipality has assumed responsibility for the maintenance of the water supply network that was constructed under SEDA. As their contribution to maintenance of this infrastructure improvement, the local residents have agreed to pay one Manat monthly for water. Local residents have themselves created mechanisms to ensure the sustainability of infrastructure development. In Amirkhanli, village residents, along a newly constructed inner road, have agreed to each be responsible for the upkeep of 100 meters of the roadway. These cases present evidence of strong sustainability prospects for program results. SEDA’s sponsored training, workshops, and other events have established and strengthened dialogue, networks, and support between the various CDCs and CDCs and GoAJ. Under Component 1, key informants interviewed, described a strong relationship between the CDCs and the municipalities in 14 19 of 15 sites visited. Component 2 and 3 training interventions often bring together GoAJ personnel with civil society counterparts but at this point it is too early to tell whether this dialogue will be sustained after project conclusion. 4.4 EVALUATION QUESTION FOUR: HOW SUCCESSFUL WAS SEDA PROJECT IN INTEGRATING GENDER CONSIDERATIONS? Findings Information from desk review, KIIs, and FGDs all indicate that the SEDA made a diligent effort to integrate gender considerations into project activities and to stress gender inclusion and equity. However, while the project was successful in integrating gender considerations into its activities, it was not able to achieve full gender equity in terms of participation. This is not surprising, given local customs and traditions that have circumscribed women’s participation in community leadership and activities. Analysis of PMP data shows that out of a total of 11,272 people participating in trainings under the three components, 36.8% are female ((see Chart 3, below). Chart 3: Overview of Gender Dimension of SEDA The SEDA team in Baku and in the three regional offices shows good overall gender balance, although female participation at the mid- or higher- levels of project management remains somewhat limited. The PMP report on “citizens benefitting from community driven projects” – estimating the number of people directly and indirectly benefiting from the Component 1 infrastructure development projects - shows that for 90 projects analyzed, of a total of 112,707 beneficiaries, 55,736 (49.5%) are female. Given the broad local participation of women and men in identifying and selecting priority infrastructure projects, and the likelihood that in these small communities infrastructure improvements will tend to directly or indirectly benefit virtually all residents, this positive finding is hardly surprising. The ET’s CDC profiles analysis shows that CDCs generally have a good representation of women, and that 15.4% of Council Heads are women. Over 50% of the CDCs have between 41- 50% women members, 31.7% have 20 - 40%, and 12.5% have over 50% women members in the structure. 20 FGDs with CDC members indicate that SEDA/UMID stressed the necessity of women’s representation and involvement in the councils and all had female representation. Not surprisingly, actual participation of women in CDCs varies between communities. FGDs revealed that selection of CDC members was often based on standing or reputation within the community, career position or skills, or simply popularity and personality, all factors affected by gender norms and traditions. The site visits found some ‘advanced’ villages with large female representation on the CDC and others with few women members. 5.0 CONCLUSIONS 1. Overall, SEDA has been conscientiously and effectively implemented within the constraints of the political context in which it has operated. • Component 1has been effectively implemented: - CDCs were established in all targeted communities. - The CDCs clearly increased citizen’s participation in community decision-making and development. - When given training opportunities and organizational support, the most remote and impoverished Azerbaijani communities are able to effectively conduct a participatory needs assessment, prioritization and project-development process. - Local SEDA-supported infrastructure projects have had a modest effect on improving the socio-economic situation in the communities. Yet, they were effective in providing community members with new skills and a sense of solidarity, empowerment, and hope in the future. • Component 2 could not be effectively implemented as designed: - While CSOs may play a valuable future role in socio-economic development, within the present political context the effectiveness of a CSO-based strategy is limited. • Component 3 has been effectively implemented, within the limitations of the current political environment: - SEDA’s contribution to the enactment of the Law on Public Participation is exemplary. - Other foci such as promotion of PPPs have not been fully implemented due to limited engagement of the GoAJ. 2. Although conceptually, complementarity and synergies between the three SEDA program components would be expected, within the complex and fraught political environment in which SEDA is actually being implemented, the potential for complementarity, particularly for Component 2, is constrained. • Some complementarity existed between SEDA’s Component 1 and 3, as demonstrated by the active participation of CDC members in awareness raising and trainings on the Law on Public Participation and the role of Public Councils. If the Public Councils mandated by the Law on Public Participation are established at the municipal level, the ET can expect CDC members to play an active role and SEDA to be well-positioned to deliver capacity building support to the Councils at all levels. • Within the current context, CSO development (Component 2) cannot be effectively integrated within the SEDA model and does not add complementarity to the other project components. 21 3. Within the immediate future, SEDA’s results and relationships are expected to be largely sustained. • Based on the continuing activities of CDCs following completion of the SEDA-sponsored infrastructure projects, many of these projects can be expected to remain functional into the near future. • CDCs may be expected to continue their current close working relationships with the municipalities after SEDA is completed. • As demonstrated in the villages visited by the ET, the small, local infrastructure projects will likely be sustained by local authorities, CDCs and community members. • The skills and knowledge acquired through SEDA’s training and the experience of community mobilization and participatory development will be retained and can be applied in addressing local development outside of SEDA’s framework. 4 Challenges remain in achieving full gender equity in SEDA program activities. • SEDA has made a diligent effort to bring gender equity to all of its activities and insisted on women’s participation in all activities including CDCs. Still, the participation of women has not been fully equitable across all of SEDA’s activities. 6.0 RECOMMENDATIONS 1. SEDA should continue to engage and provide support to the GoAJ within the purview of its SOW, and strengthen linkages and complementarity of these activities with its local socio-economic development initiatives. 2. Cluster projects may offer an opportunity for mobilized communities to cooperatively engage in addressing larger socio-economic priorities. As such, they should be further explored by SEDA. 3. SEDA/USAID should consider the potential benefits and trade-offs of increasing the size of SEDA’s infrastructure grants. 4. Resources permitting, the current Component 1 model of CDC development could be productively expanded to additional regions of Azerbaijan. 5. Should in the present economic context, USAID/Azerbaijan decide to focus on distinct economic development objectives, traditional program models of entrepreneurship training and support, SME and value-chain development must be considered in conjunction with the MoEI. 6. Assuming the creation of Public Councils in the near future: • Assistance should be provided to CDC members in transitioning to roles as members of local Public Councils. • Training support should be offered to Public Councils and assistance provided to the GoAJ in establishing monitoring and accountability systems for Councils at all levels. 7. If establishment of Public Councils is delayed, further assistance to CDCs is advisable to support their sustainability and maintain community morale. 8. At this stage of SEDA’s implementation, if there is improvement in the enabling environment, a CSO-based community development strategy should be created as a distinct program rather than be incorporated into SEDA. 22 9. SEDA should work with all participating CDCs to create locally-appropriate sustainability plans to facilitate on-going CDC activity following project completion. 10. During its final program year SEDA should work with its GoAJ counterparts to facilitate continued dialogue, consultation and engagement of community members in local decision-making on socio￾economic development. 11. SEDA should set specific targets for increased women’s participation in all subsequent activities, including training events, conferences and study tours, and hold staff accountable for their achievement. 12. SEDA should consider conducting specific skills-building training and support activities for female CDC members. 13. Given the current momentum and upturn in program activities, if SEDA funds remain after the fifth and final year of the project, a no-cost extension should be considered to maximize program results. 7.0 LESSONS LEARNED 1. SEDA’s success in facilitating the establishment and capacity-building of CDCs in more than 100 small and often remote and impoverished villages, and the communities’ enthusiasm and competence in planning, supporting and completing small but often strategic infrastructure development projects, validates its vision of local collaboration and participation. 2. The development context may prove contradictory, presenting unlikely opportunities as well as constraints. While a complex and fraught political environment may limit effective pursuit of some program objectives and activities - as was the case for CSO-development under Component 2 – space was open for the achievement of other related activities, particularly SEDA’s support for the enactment of the Law on Public Participation. 3. SEDA’s design, which combines three interrelated components, provides a kind of robustness and resiliency enabling the program to adapt and shift areas of emphasis, as necessary. 4. Despite a discouraging and, at times, threatening context for program implementation, rather than close down, SEDA/EWMI and USAID/Azerbaijan demonstrated that a commitment to development, adaptability, and flexibility, based on relationship-building and local knowledge, can enable a program to successfully reconfigure and focus on areas of opportunity while maintaining its value and integrity. 5. SEDA has demonstrated the utility of rooting civic engagement, participation and citizen/governmental collaboration in small-scale socio-economic development activities. Over time, however, local and central government and the private sector will have to find a mechanism to further engage and work with local communities to address more substantial and social development needs, a prospect made more challenging by the current economic crisis. 1 ANNEXES 2 Annex 1. Scope of Work 3 STATEMENT OF WORK Performance Evaluation of the Socio-Economic Development Activity (SEDA) July 21, 2015 I. SCOPE The Contractor shall provide non-personal services for a performance evaluation of the following Project: Project Title: Socio-Economic Development Activity (SEDA) Contract Number: AID-112-A-11-00002 Period of Performance: September 30, 2011 to September 29, 2016 Total Estimated Cost: $14,399,462 Contractor: East-West Management Institute Period to be evaluated: September 30, 2011 – September 30, 2015 II. PURPOSE USAID/Azerbaijan seeks to carry out a Performance Evaluation of the implementation of the Socio￾Economic Development Activity (SEDA). The project has been under implementation since September 2011 and will end on September 29, 2016. The purpose of the performance evaluation of SEDA is to: 1. Determine the effectiveness of each component and their complementarity to each other; 2. Provide recommendations to improve project implementation. The evaluation must cover the project implementation period of September 30, 2011 - September 30, 2015. The primary audience for the SEDA Project performance evaluation is the USAID/Azerbaijan Mission. Other audiences include USAID/Washington (Bureau for Europe and Eurasia (EE/DGST), Bureau for Democracy, Conflict and Humanitarian Assistance (DCHA), Bureau for Policy, Planning, and Learning), East-West Management Institute, Government of Azerbaijan (Ministry of Economy and Industry, Council for State Support to NGOs), and other USAID implementing partners and beneficiaries of SEDA Project activities (CSOs, communities). The Mission may share the final evaluation report with other donors, other implementing partners, host Government counterparts and other United States Government (USG) agencies operating in Azerbaijan. The Mission will use the evaluation findings to guide SEDA’s continuing implementation. III. A. BACKGROUND In 2011, USAID entered into a Cooperative Agreement with the East-West Management Institute in the amount of $14,399,462 in order to increase civic participation in governance at local and regional levels, using consultation and collaboration processes between citizen groups and local government that address social and economic needs in selected regions of Azerbaijan through community mobilization, civil society engagement and support to local authorities. 4 This 5-year Activity is co-financed by the Government of the United States and the Government of Azerbaijan and reflects the parties’ strong intention to cooperate under the Economic Partnership Commission based on the intergovernmental MoU signed in 2009, and the MoU signed between USAID and the Council for State Support to Non-governmental Organizations under the President of the Republic of Azerbaijan. This evaluation must review specific aspects of three components of SEDA in relation to the achievement of this ultimate objective: Component 1 - Revitalization or establishment of village and/or district-based Community Development Councils (CDCs) to increase community participation in socio￾economic development at the local and regional levels; Component 2 - Strengthening the capacity of Civil Society Organizations (CSOs) to engage with citizens and government officials on local, regional, and national socio-economic priorities; and Component 3 - Strengthening the capacity of the Government of Azerbaijan (GOAJ) to engage with stakeholders on key socio-economic development issues. Through this activity USAID will mobilize and develop capacity of up to 100 local communities in three regions of Azerbaijan (Ganja-Gazakh, Guba-Gusar and Aran) to learn the basics of citizen participation as well as identify, design and implement up to 140 small infrastructure projects. USAID will engage regional CSOs into connecting citizenry and authorities in solving local socio-economic problems and improve the capacity of local authorities to effectively engage with communities and other stakeholders on socio-economic development issues. This activity will also assist the relevant governmental agencies improve civil society-related legislation to provide more space and mechanisms for meaningful citizen participation. Azerbaijan continues to experience strong economic growth. Nonetheless, local governance is poorly developed and citizens realize little if any connection between their actions or the activities of local government to improve their living standards. Azerbaijan maintains a strong connection to a post-soviet system of governance that depends almost entirely on central control. Part of this legacy system is the appointment of regional governors (ExComs) who are the appointed representatives of the president. These local authorities are the foci of all governance in the regions, which reinforces the centralization of government and discourages the formation of local citizen participation and the development of effective local government structures. Municipal structures are in place, but enjoy very limited power and have insignificant funds and fund-raising leverages. Consequently, Mission funds which support civil society development in the country, among others, are geared towards building and sustaining bridges between citizenry and local authorities. Despite the steady decline of funding over the past several years, the Mission is very committed in supporting local formal and informal civil society institutions and private sector partners that serve as engines of growth and progress for their own people. The SEDA strategy is aligned well into the USAID/Azerbaijan Mission’s Country Development Cooperation Strategy for 2011-20161 where one of the three main Intermediate Results (IR 2.2 Civil society strengthened to increasingly engage in development and political processes) under the Development Objective 2 (Effective participation of diverse actors and institutions in the democratic development of Azerbaijan increased) is set to ensure a more participatory environment through strengthening non-governmental actors and organizations to have more influence with GOAJ and policymakers on Azerbaijan’s development. 5 Fortunately, there is still space for civil society activities in Azerbaijan and USAID’s objective is to expand that space in meaningful ways. As a result of the context and environment, the Mission designed the Socio-Economic Development Activity (SEDA) in 2010 to increase the breadth, depth and effectiveness of local actors and institutions while fostering dialogue and cooperation between GOAJ and civil society. The activity became a part of a broader Memorandum of Understanding between the USG and the GOAJ, eventually turning into a large co-financed program under the Economic Partnership Commission established between the two governments. The activity is in line with the current strategy of the GOAJ to develop regions of the country indicated in the State Program for Poverty Reduction and Sustainable Development for 2008-20152. B. IMPLEMENTATION PROGRESS TO-DATE: • Under Component 1, the implementing partner mobilized 57 communities in three project regions, conducted 572 trainings for Community Development Councils on the following topics: organizing meaningful meetings, decision-making processes, conducting needs assessments, stakeholder analyses, basics of cost/benefit analysis, assessing environmental impact, project identification, and design and implementation. To date, the communities have completed 28 small infrastructure projects, are implementing 10 others, have an additional 16 projects under contract preparation and an additional 15 in the pipeline. The first National Conference on Community Development with 120 participants from SEDA-supported communities was held in Ganja city on May 13, 2015. • Under Component 2, the Implementing Partner provided 6 Civic Engagement Grants (CEG) to local NGOs, who conducted advocacy projects on various socio-economic issues. SEDA assisted 2 Regional NGO Resource and Training Centers with organizational assessment, and preparation of organizational manuals on Finance and Operational HR and Procedures. In total, 13 trainings with participation of 303 people were conducted on advocacy, financial planning, public relations, proposal writing, project management, and NGO management. • Under Component 3, the Implementing Partner provided a series of recommendations to improve the Public Participation Law, prepared an Overview of Mechanisms of Public Participation in Azerbaijan, printed a handbook on International Best Practices on Citizen Participation, and developed the Manual on Establishment of Public Councils. Government officials participated in several trainings, including the Annual Advanced Seminar for Participatory Policy Development, and Monitoring & Evaluation training. The “Public Participation: Legal and Practical Issues” National Conference held in Ganja on November 13, 2014, launched the work on establishment of Public Councils. To raise awareness on establishing Public Councils, the Implementing Partner held 3 regional Round Tables and 33 workshops covering all SEDA target regions through December 2014 – February 2015. SEDA held the introductory Workshop on Public Private Partnerships (PPP) on April 28-30, 2015 for 50 representatives from multi-sector organizations interested in the development and implementation of PPP projects in Azerbaijan. IV. EVALUATION QUESTIONS In order to answer the questions below related to SEDA performance, the Contractor must review pertinent background documents relating to SEDA and the broader civil society context in Azerbaijan, as well as conduct interviews in Washington, DC with field work in Azerbaijan for four weeks. Among other methodologies, the Contractor must collect information from primary sources such as Azerbaijani counterparts, the beneficiaries of the assistance, CSOs involved with SEDA, and other donors and implementing partners. 6 Evaluation questions include the following: 1. How effective has the implementation of each component been? 2. How complimentary were the activities under the three components identified in the scope of the project towards each other? Which interventions were the most effective towards achieving complementarity of results? 3. How sustainable are the established dialogue and relationships between CDCs, other civil society and government officials? 4. How successful was SEDA project in integrating gender considerations? V. EVALUATION DESIGN AND METHODOLOGY The research design must be an integral part of each proposal, and must explain methodologies in detail that will be used to collect required information. For evaluations, the design must outline in details what methods the Contractor will use to get answers for each evaluation question. USAID/Azerbaijan recommends using both quantitative and qualitative methods (data, observations and qualitative insights/evidence) in assessing progress, results and impacts. The Contractor must review the Agreement, annual work plans, quarterly performance and financial reports, baseline data on established indicators, results/outcomes and annual targets. The Mission recommends that the evaluators use rapid appraisal methods, face-to-face interviews with key informants, focus groups, group discussions, and community discussions, as appropriate, to assist in answering the above questions and also to gauge satisfaction by the government and key stakeholders about project performance. The Contractor is encouraged to employ other methods as long as they do not add to the duration or cost of the evaluation. The Contractor must review the documents cited in the Reference Documents section of this Statement of Work. These documents provide the context of the civil society development in Azerbaijan and information on SEDA’s work over the years: the management, implementation, monitoring, and reporting of activities and their financing. The Contractor must include in their proposal the proposed data collection methodology to answer the evaluation questions and draft work plan. The draft evaluation design must include detailed evaluation matrix (including key questions, methods and data sources used to address the questions and the data analysis plan for each question), draft questionnaires and other data collection instruments or their main features, known limitations to the evaluation design, a work plan, and a dissemination plan. The draft evaluation design must also include specific sub-questions for each evaluation question. The draft work plan must include a schedule, logistical arrangements, and a proposed evaluation team with clearly defined roles and responsibilities for each member. VI. TASKS 7 B. Review of Key Documents: The Contractor must review key documents to develop a Work Plan prior to any field work. All available documentation describing SEDA Project activities carried out in Azerbaijan must be reviewed. Documents for review include but are not limited to those listed in the reference section. The Contractor must contact the designated Agreement Officer’s Representative (AOR) for the SEDA Project evaluation for access to relevant documents. C. In-briefing: Upon arrival in Azerbaijan, the Evaluation Team must provide an entrance briefing to the designated USAID officials, introduce the team, discuss logistics and scheduling, discuss submission of the Work Plan, and any other issues. USAID will assist with identification of the relevant stakeholders to meet with and provide additional suggestions for interviews. D. Work Plan: The Work Plan must be in accordance with the USAID prepared timeline for all work to be concluded and the dates for submission of the draft and final reports. The Work Plan must include the following elements: I. Schedule of contacts and site visits (regions, beneficiaries and collaborators); II. Arrangements for local logistics; III. Schedule of briefings and submission of deliverables; and IV. Delineate the roles and responsibilities of the other members of the Evaluation Team to ensure coverage of all elements of the Statement of Work. USAID will provide comments within two days. E. Evaluation Design: The Contractor must prepare and submit to USAID for approval a final evaluation plan and schedule with the following major elements: Schedule, methodology for conducting the evaluation (data and information collection instruments, field interviews, participant surveys), beneficiary groups to be contacted and regions (Ganja-Gazakh, Guba-Gusar and Aran) to be visited provided by USAID/Azerbaijan. The Contractor must ensure that its findings and conclusions about the effectiveness of the SEDA Project activities are based on available data that is both accurate and reliable, and that information gathered is representative of and reasonably reflects results actually achieved. The Contractor must submit a final detailed evaluation design, which must consist of the following: I. List of topics and relevant questions, methods and data sources for data gathering; II. A matrix of regions and beneficiaries to be contacted; III. Data analyses for each question and presentation plan; IV. Data collection instruments; and V. Limitations of the evaluation design, if any. The Evaluation Team must share the evaluation design with the Implementing Partner for comment, but, in the interest of objectivity and independence, the Implementing Partner will not participate in the implementation, analysis, or presentation of the evaluation. F. Field Work: The Contractor must begin field work after finalization of the Work Plan and Evaluation Design and its approval by USAID. 8 G. In-country USAID Debrief: The Contractor must provide an oral debriefing to USAID upon completion of the evaluation and prior to departing from Azerbaijan. Evaluation findings must include facts, evidence and data. Recommendations must be specific, concise and supported by evidence. Recommendations must have clear evidence linkages, based on credible conclusions, that are in turn directly linked to evaluation findings. The “evidence continuum” (findings→ conclusions→ recommendations) should be followed. Recommendations must be action-oriented and implementable. H. Evaluation Report: I. The Contractor must provide USAID with a detailed Outline of the Evaluation Report, main findings, and recommendations to USAID prior to departing Azerbaijan. II. The Contractor must submit a draft report of its findings within ten (10) work days after departing Azerbaijan for review and comments. The first draft must address comments and recommendations made by USAID during the out-briefing. III. The Contractor must submit the final evaluation report to USAID five (5) days after receipt of comments. The final report must be concise, well written, and comprehensive. Recommendations must be action-oriented, practical, and specific; define responsibilities and timelines for the action; and identify milestones and deliverables. Unresolved issues that highlight what remains to be done must also be included in the final report. VII. DELIVERABLES The Contractor must submit the following deliverables to USAID within the terms defined in the contract: A. Evaluation Work Plan and Design: 1. Evaluation Work Plan and Schedule: The performance evaluation Work Plan must be submitted to USAID for review before departure to Azerbaijan. 2. Evaluation Design: The evaluation design must be submitted to USAID for review before departure to Azerbaijan. USAID will provide its comments to the Evaluation Work Plan and Evaluation Design within two work days respectively. The Contractor must submit the Evaluation Work Plan and Evaluation Design, with USAID comments incorporated, within a day upon receipt of USAID comments for approval. B. Draft Evaluation Report: The Contractor must provide USAID with a first draft of the written report within ten (10) work days after the departure from Azerbaijan. C. Final Evaluation Report: The Evaluation Team Leader will have an additional five (5) working days from the date of receipt of USAID comments by the Contractor to finalize the report. The report (excluding annexes) must not exceed 25 pages. The Contractor must submit the final evaluation report to USAID no later than January 25, 2016. The Final Evaluation Report format must be presented in Microsoft Word and use 12-point type font throughout the body of the report, using page margins 1” top/bottom and left/right. The body of the report must not exceed 25 pages, excluding the executive summary, table of contents, references and annexes. The final report must follow USAID branding and marking requirements. 9 The Final Evaluation Report must include an executive summary, introduction, the development context and the background of the project being evaluated, evaluation questions, explanation of evaluation methodology, the limitations of the evaluation, findings, conclusions, lessons learned thus far, and recommendations on the directions and adjustments (if any) for successful implementation of SEDA Project activities. The executive summary must summarize the purpose, background of the project being evaluated, evaluation questions, evaluation methodology, major findings, lessons learned, conclusions, and recommendations. The evaluation methodology must be explained in the report in detail. Limitations to the evaluation must be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.). The annex to the report must include: 1. Evaluation Statement of Work; 2. Schedule of Evaluation; 3. Evaluation design/methodology employed questionnaire and list of questions by topic, etc.; 4. Names and contact information of key respondents, sites visited and other sources of information, properly identified and listed; 5. Information, as appropriate, regarding significant unresolved issues, difference of opinions (among members of the evaluation team, the Implementing Contractor, beneficiary CSOs, Government counterpart(s) and other relevant stakeholders) and availability of data and its quality; and 6. Evaluation Design. Per the USAID evaluation policy, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report. • The evaluation report should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate what worked in the project, what did not, and why. • Evaluation reports must address all evaluation questions included in the scope of work. • The evaluation report must include the Statement of Work as an annex. USAID/Azerbaijan must agree in writing with all modifications to the Statement of Work, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline. • The evaluation methodology must be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides must be included in an Annex in the final report. • The evaluation findings must ensure that, where relevant, data analysis takes gender into consideration. • Limitations to the evaluation must be disclosed in the report, with particular attention paid to the limitations associated with the evaluation methodology. 10 • The evaluation findings must be presented as analyzed facts, evidence, and data. Findings must be specific, concise and supported by strong evidence. • Sources of information need to be properly identified and listed in an annex. • Recommendations need to be supported by a specific set of findings. • Recommendations must be action-oriented, practical, and specific, with defined responsibility and timelines for the action. D. Interview Notes and Other Relevant Records: The team must submit all interview notes and other relevant records to USAID in an electronic file in an easily readable format in English. All data, records, and reports from the evaluation are owned by USAID. The Contractor must submit the final report to USAID’s Development Experience Clearinghouse (DEC) within 30 days following USAID approval. VIII. EVALUATION SCHEDULE The illustrative schedule is as follows: Documents review: November 16-20, 2015 Draft evaluation design and a work plan submitted to USAID: November 20, 2015 In-brief at USAID: November 23, 2015 In-country work: November 23 - December 14, including visits to districts Out brief at USAID and outline of the final report: December 14, 2015 Draft Final Report due to USAID: December 30, 2015 USAID comments due to the Contractor: January 15, 2016 Final report: January 25, 2016 Table 1 below shows the illustrative timeline for each activity. Table 1. SEDA Project Performance Evaluation Mon Tue Wed Thu Fri Sat Sun Pre-Design Activities Week 1 Nov 16 Nov 17 Nov 18 Nov 19 Nov 20 Nov 21 Nov 22 Review Documents Conduct Washington Interviews Submit Draft Evaluation Work Plan and Draft Evaluation Design Travel to Baku Baku Activities and Review Week 2 Nov 23 Nov 24 Nov 25 Nov 26 Nov 27 Nov 28 Nov 29 In Brief with USAID USAID Comments on Evaluation Work Plan and Evaluation Design Submit Evaluation Work Plan and evaluation design with USAID comments incorporated Interview AOR/alternate AOR Interview SEDA Project 11 Interview Donors Meet GOAJ officials Write Notes up Field Activities Weeks 3 and 4 Nov 30 Dec 1 Dec2 Dec 3 Dec 4 Dec 5 Dec 6 Travel to field Field Interviews Dec 7 Dec 8 Dec 9 Dec 10 Dec 11 Dec 12 Dec 13 Return to Baku Interim briefing to USAID after field interviews Final interviews in Baku Team write up notes/Development of detailed outline of main findings and recommendations Baku Activities Week 5 Dec 14 Dec 15 Dec 16 Dec 17 Dec 18 Dec 19 Dec 20 Debrief USAID Evaluation Team departs Baku Team Leader writes Draft Report Write report Week 6 Dec 21 Dec 22 Dec 23 Dec 24 Dec 25 Dec 26 Dec 27 Team Leader writes draft report Holiday Week 7 Dec 28 Dec 29 Dec 30 Dec 31 Jan 1 Jan 2 Jan 3 Team Leader writes draft report Draft report submitted to USAID Holiday Holiday Week 8 Jan 4 Jan 5 Jan 6 Jan 7 Jan 8 Jan 9 Jan 10 USAID reviews the draft report Week 9 Jan 11 Jan 12 Jan 13 Jan 14 Jan 15 Jan 16 Jan 17 USAID reviews the draft report USAID provides comments Week 10 Jan 18 Jan 19 Jan 20 Jan 21 Jan 22 Jan 23 Jan 24 Team Leader revises the draft report and submits the final report Holiday Week 11 Jan 25 Team Leader revises the draft report and submits the final report 12 IX. LEVEL OF EFFORT The following levels of effort are illustrative and should serve only as an example of the staff which may be mobilized under this Task Order. These levels may not reflect the actual level of effort contracted, and the Contractor will be expected to submit its own estimate of the level of effort needed to fulfill the objectives. Illustrative level of effort: Total Number of days in Country/Consultant No. of Work Days in Country/ Consultant Travel Days Total No. of Work Days/Consultant Evaluation advisor – Senior Team Leader 24 20 4 39 Technical Advisor – Mid￾level 24 20 4 39 Local Consultant 20 Local Translator/Administrative Assistant 18 It is possible that the level of effort will not be equally distributed among experts. X. REPORTING RELATIONSHIP The USAID/COR will provide overall direction to the Evaluation Team, identify key documents and primary contacts. XI. LOGISTICS The Mission will facilitate meetings with the beneficiary communities, host country counterparts, other relevant stakeholder/partners and provide overall direction to the Evaluation Team. The Contractor must arrange for local transportation and for making travel arrangements within Azerbaijan as required, and cover all payment/funds outlay for these services. The Contractor is ultimately responsible for ensuring that its logistics needs are met. USAID personnel may accompany the Evaluation Team as observers (as appropriate) on meetings in Baku and in the regions. USAID will provide its own logistics to be present at these meetings. USAID personnel may join the Evaluation Team during the design, implementation and presentation phases. XII. DOCUMENTS FOR REVIEW To the extent possible, relevant reports and other project documentation that have not been posted to the DEC will be provided by the Mission to the Contractor prior to travel to Azerbaijan. These documents are: 13 • Program Description as stated in the Cooperative Agreement; • Implementing partner’s Quarterly Reports; • Initial list of in-country contacts; • PMP indicator tables; • M&E plans submitted and approved by USAID; • Other deliverables (expert report, publications) produced by the Implementing Partner • List of beneficiaries XIII. FINAL REPORT FORMAT The final report format must be presented in Microsoft Word and use 12-point type font throughout the body of the report, using page margins 1” top/bottom and left/right. The body of the report must not exceed 25 pages. XIV. OTHER REQUIREMENTS All quantitative data collected by the evaluation team must be provided in an electronic file in easily readable format agreed upon with the Agreement Officer’s Representative (AOR). The data should be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed. All modifications to the SOW, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline, need to be agreed upon in writing by the AOR. 14 Annex 2. Work Plan and Design 1 1.0 Introduction The Socio-Economic Development Activity (SEDA) Project is a five-year initiative being implemented by the East-West Management Institute (EWMI) and local partners that seeks to enhance civic participation in local and regional governance, enhance the capacity of civil society and local governing authorities to productively consult and collaborate on identifying and addressing local socio-economic development priorities. The performance evaluation of the SEDA Project seeks to determine the effectiveness and complementarity of SEDA’s three core program components during the four year period from 30 September 2011 to 30 September 2015, and to provide recommendations to improve project implementation and results. The evaluation is organized to address USAID/Azerbaijan’s four key questions: 1) how effectively has each component been implemented; 2) how complementary are the activities addressing each component and which interventions were the most effective towards achieving complementarity of results; 3) how sustainable are the dialogue and relationships established between community development councils (CDCs), and other civil society and government officials; and 4) how successful is SEDA in integrating gender considerations? The SEDA performance evaluation has been designed as a participatory process, enabling the ME&A evaluation team to actively engage USAID staff in Azerbaijan and other stakeholders through all phases of the evaluation. Participatory evaluations explicitly incorporate beneficiaries and stakeholders’ views of program’s implementation, synergies, outcomes and sustainability which enhance the validity and utility of findings, recommendations and conclusions, can help build consensus and help identify and explain any areas of difference. The evaluation design is multidimensional, employing an array of complementary assessment techniques: quantitative and qualitative data collection; individual and group responses; focus group discussions (FGDs) and surveys; direct observations; and community discussions, among others. In addition, to assess and empirically demonstrate complementarities and synergies among SEDA’s three components, we will also explore process tracing technique, an evidenced￾based qualitative analytical approach that uses detailed program description to generate and test alternative causal inferences and hypotheses for observed change. The above techniques will help us capture the diversity of opinions and perceptions of beneficiaries and stakeholders about the achievements, effectiveness, and sustainability of SEDA. They will also uncover unexpected positive or negative impacts of the project’s interventions on strengthening citizen participation, improving the capacity of local authorities to effectively engage with communities and other stakeholders, and assisting governmental agencies to improve civil society legislation. In light of the changes in the political and socio-economic context over SEDA’s four-year implementation period, the evaluation will also consider relevant exogenous factors over time, to provide a more nuanced understanding of the relationship between implementation of activities, constraining and facilitative factors in the environment, and program outcomes and development sustainability. To do this, the evaluation will compare communities engaged earlier in program implementation with those that participated later in the process in order to provide insights into program maturation, changes, and sustainability of results. SEDA is a complex, holistic developmental intervention, implemented over multiple years at multiple levels, in scores of local communities across three regions. Recognizing SEDA’s complexity, we will examine the program at both a micro and macro level. At the micro level, we will generate detailed descriptive and analytical quantitative and qualitative evidence necessary to address the full array of detailed evaluation sub-questions indicative of program implementation and performance. At the macro level, we will combine these findings to 2 identify over-arching patterns and themes in the program as a whole, and address the four large, primary evaluation questions. As gender represents a particularly important variable, the design enables us to pay particular attention to sex-disaggregated data and observe gender-sensitive indicators in evaluating SEDA’s outcomes. If gender-based differences are noted, we will seek to identify the factors underlying such results and recommend options for improving equitable program performance and results. Given the inherent limitations in time for data collection and in sample selection and size, during data analysis we will use the triangulation method in which we draw evidence from the full array of qualitative and quantitative methods we have deployed to maximize the validity and utility of the findings and clearly discern any areas of uncertainty or disagreement. 2.0 Evaluation Design The SEDA performance evaluation has been designed to obtain - from various stakeholder sources and utilizing an overlapping variety of assessment instruments and techniques – evidence sufficient to satisfactorily address USAID/Azerbaijan’s four key evaluation questions, to document program results, and to recommend plausible measures to enhance program performance, outcome, and sustainability. To ensure that information collection will serve to adequately answer the key questions, the ME&A team have created a matrix of sub-questions that provide an overall structure to the inquiry and serve to generate the specific data items that together act as indicators for each major question. In sum, and contextualized within the macro themes defining the socio-economic environment in Azerbaijan, addressing these sub- questions will provide the empirical basis for the evaluation findings and recommendations. 2.1 Evaluation Design Matrix The SEDA evaluation matrix is presented below: SEDA Evaluation Matrix12 Evaluation Question Data Sources Instruments Notes How effective has been the implementation of each component? 1.1 To what extent were local/village-based CDCs established or revitalized? 1.1.2 Has meaningful citizen participation in local socio-economic development activities increased? Key informant interviews Surveys Focus group discussions Narrative success stories Key informant interview protocol Survey questionnaire Focus group discussion protocol 1.1.3 Do CDCs enable citizens have a greater voice in determining local development needs and priorities? Key informant interviews Surveys Focus group discussions Narrative success stories Key informant interview protocol Survey questionnaire Focus group discussion protocol 1.1.4 Do local citizens better understand their roles and responsibilities in local governance? Key informant interviews Surveys Focus group discussions Narrative success stories Key informant interview protocol Survey questionnaire Focus group discussion protocol 1.1.5 Have local infrastructure improvement projects been designed, implemented or completed? Project document review Key informant interviews Surveys Focus group discussions Narrative success stories Key informant interview protocol Survey questionnaire Focus group discussion protocol 1.1.6 To what extent do local governmental authorities understand and accept citizen participation in local governance? Key informant interviews Surveys Key informant interview protocol Survey questionnaire 12 The evaluation matrix will be updated as warranted following USAID/Azerbaijan feedback. Evaluation Question Data Sources Instruments Notes 1.1.7 Do what extent do local authorities appear to support the meaningful role of CDCs at the local/regional levels. Key informant interviews Surveys Focus group discussion Key informant interview protocol Survey questionnaire 1.1.8 Are there gender-based different rates and roles of participation in CDCs? Project document review Also addresses question 4. 1.1.9 Are there gender-based differences in leadership and participation in local infrastructure improvement initiatives? Project document review Key informant interviews Key informant interview protocol Also addresses question 4. 1.1.10 Do local citizens believe that CDCs offer them a fair and meaningful role in community governance? Focus group discussions Key informant interviews Focus group discussion protocol Key informant interview protocol 1.1.11 Do participants believe that the training they received equipped them with useful knowledge and skills for community engagement and development? Focus group discussions Focus group discussion protocol 1.1.12 Which trainings and specific skills have been most/least useful? Focus group discussions Key Informant interviews Project document review Focus group discussion protocol Key informant interview protocol 1.1.13 Did the National Conference on Community Development strengthen the operations or outcomes of CDCs? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol 1.1.14 What changes in the project plan or implementation would improve the results of the project work with CDCs? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol 1.1.15 Were the activities comprising Component 1 implemented according to plan and on schedule? Project document review 1.1.16 Were any changes in Component 1 activities implemented over the course of the project? Project document review Evaluation Question Data Sources Instruments Notes 1.2 Has SEDA strengthened the capacity of CSOs to engage citizens and government officials on local, regional and national socio-economic priorities? 1.2.1 What criteria were employed in selecting CEG grantees? Key informant interviews Project document review Key informant interview protocol 1.2.2 What specific socio-economic advocacy activities were conducted under each grant? Project document review Key informant interviews Key informant interview protocol 1.2.3 To what extent were local community members engaged in designing the local advocacy activities? Key informant interviews Project document review Focus group discussion Narrative success stories Key informant interview protocol Focus group discussion protocol 1.2.4 What targets were established for each activity? Project document reviews 1.2.5 To what extent did women participate in the planning and implementation of the activities? Project document review Key informant interviews Focus group discussions Narrative success stories Key informant interview protocol Focus group discussion protocol Also addresses question 4. 1.2.6 Who are the beneficiaries for each activity? Project document review Key informant interviews Focus group discussions Narrative success stories Key informant interview protocol Focus group discussion protocol Also addresses question 4. 1.2.7 To what extent were these activities judged to be useful and effective? Project document review Key informant interviews Focus group discussions Narrative success stories Key informant interview protocol Focus group discussion protocol 1.2.8 What changes to the grant process or the implementation of grant activities would improve performance or results? Key informant interviews Focus group discussions Narrative success stories Key informant interview protocol Focus group discussion protocol 1.2.9 What was the nature of SEDAs assistance to the Regional NGO Resource and Training Centers? Project document review Key informant interviews Key informant interview protocol 1.2.10 Did this assistance serve to increase the capacity or Key informant interviews Key informant interview protocol Evaluation Question Data Sources Instruments Notes operations of the centers? 1.2.11 How do participants rate the quality and usefulness of the NGO capacity-building training SEDA delivered? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol 1.2.12 What changes in the project plan or implementation would improve the results of the project work with CSOs and the Regional Centers? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol 1.2.13 Were the activities comprising Component 2 implemented according to plan and on schedule? Project document review Key informant interviews Key informant interview protocol 1.2.14 Were any changes in Component 2 activities implemented over the course of the project? Project document review Key informant interviews Key informant interview protocol 1.3Has SEDA strengthened the capacity of GOAJ to engage with stakeholders on key socio-economic development issues? 1.3.1 What key activities were implemented in Support of Component 3? Project document review Key informant interviews Key informant interview protocol 1.3.2 Is there evidence that the SEDA written products on Public Participation law, citizens’ participation and Public Councils have contributed to increased GOAJ engagement with stakeholders on socio-economic issues? Project document review Key informant interviews Narrative success stories Key informant interview protocol 1.3.3 Have any of the SEDA recommendations on improving Public Participation Law been adopted? Key informant interviews Document review Key informant interview protocol 1.3.4 How useful did GOAJ personnel find the SEDA sponsored training workshops in increasing their capacity and willingness to engage stakeholders? Key informant interviews Key informant interview protocol 1.3.5 Is there evidence that the SEDA training events have contributed to increased GOAJ engagement with stakeholders on Project document review Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol Evaluation Question Data Sources Instruments Notes socio-economic issues? Narrative success stories 1.3.6 Do participants in the regional Round Tables report an increased awareness on establishing Public Councils as a result of this activity? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol 1.3.7 Do participants in the workshops on establishing Public Councils demonstrate an increased awareness as a result of this activity? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol 1.3.8 Is there any evidence to suggest an increased capacity or willingness of GOAJ personnel to engage with stakeholders as a result of the Round Tables and training events? Key informant interviews Focus group discussions Narrative success stories Project document review Key informant interview protocol Focus group discussion protocol 1.3.9 To what extent have participants in the PPP workshop increased their knowledge, capacity or willingness to implement PPP Projects? Key informant interviews Focus group discussions Narrative success stories Project document review Key informant interview protocol Focus group discussion protocol 1.3.10 What is the gender breakdown for participants in the various trainings implemented under Component 3? Project document review Also addresses question 4. 1.3.11 What changes in the project design or implementation plan would increase GOAJ capacity and willingness to engage with stakeholders on socio-economic development issues? Key informant interviews Project document review Key informant interview protocol 1.3.12 Were the activities comprising Component 3 implemented according to plan and on schedule? Project document review 1.3.13 Were any changes in Component 3 activities implemented over the course of the project? Project document review 1.4 Did exogenous factors in the political and economic environment of Azerbaijan over the course of the project affect implementation of the three components? 1.4.1 What factors or events in the political and economic Key informant interviews Document review Key informant interview protocol Evaluation Question Data Sources Instruments Notes environment may have affected SEDA project implementation? 1.4.2 What external factors if any, may have facilitated effective implementation? Key informant interviews Key informant interview protocol 1.4.3 What factors if any, may have constrained effective implementation? Key informant interviews Key informant interview protocol 1.4.4 How successful was SEDA in recognizing and mitigating constraining factors? Key informant interviews Project document review Key informant interview protocol 2.0 How complementary were the activities under the three components? Which interventions were most effective towards achieving complementarity of results? 2.1 Is there evidence of attempts at complementarity amongst components in the SEDA design, PMP. Logic model, quarterly reports or other project documents? Project document review 2.2 Can SEDA project staff site specific examples of complementarity or synergies in project design, delivery or outcomes? Key informant interviews Survey Key informant interview protocol Survey questionnaire 2.3 To what extent are SEDA beneficiaries aware of the overall program design and of components in which they have not directly participated? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol Community fora protocol 2.4 Do key informants including the AOR, donors in the area of community participation and socio-economic development, and knowledgeable civil society actors identify plausible complementarities in the implementation of activities across components? Key informant interviews Survey Key informant interview protocol Survey questionnaire 2.5 Do key informants rate particular activities or interventions as having demonstrated greater complementarity in supporting or Key informant interviews Survey Key informant interview protocol Survey questionnaire Evaluation Question Data Sources Instruments Notes facilitating the implementation of other interventions? 2.6 Is evidence available to support causal inferences affirming component complementarity? Project document review Key informant interviews Key informant interview protocol 2.7 Can program or CSO staff, key stakeholders or program beneficiaries provide narrative accounts or success-stories indicative of complementarity? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol Community fora protocol Network mapping protocol 2.8 Do references to complementarity appear in FGDs? Focus group discussion Focus group discussion protocol 3.0 How sustainable are the established dialogue and relationships between CDCs, other civil society structures and actors and GOAJ officials? 3.1 Of 57 engaged communities, how many have CDCs which have met within the past 60 days? Project document review Focus group Discussions Key informant interviews Focus group discussion protocol Key informant interview protocol 3.2 How do CDC, CSO and GOAJ officials rate the sustainability of their linkages? Key informant interviews Surveys Key informant interview protocol Survey questionnaire 3.4 Do any narratives suggest sustainability of local and regional linkages? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol Community fora protocol 3.5 Do any success stories indicate sustaining local and regional linkages? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol Community fora protocol 3.6 How do SEDA staff members rate the sustainability of program linkages? Key informant interviews Key informant interview protocol 3.8 Do community fora participants express belief that linkages will Focus group discussions Community fora protocol Evaluation Question Data Sources Instruments Notes be sustained? 3.9 Have any MOU or other agreements been executed indicating sustaining linkages? Project document review Key informant interviews Key informant interview protocol 3.10 Do CSO and local governance experts believe that the SEDA local linkages will be sustained? Key informant interviews Key informant interview protocol 3.11 Do examples exist of communities sustaining the program linkages following completion of SEDA inputs/support? Project document review Key informant interviews Focus group discussions Program narratives Key informant interview protocol Focus group discussion protocol Network mapping protocol Community fora protocol 3.12 Are there any examples of other donors supporting further socio-economic development in SEDA target communities? Project document review Key informant interviews Key informant interview protocol 4.0 How successful was the SEDA project in integrating gender considerations? 4.1 Do women and men differ in their assessment of SEDA’s openness to their participation? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol Community fora protocol 4.2 Does the SEDA design or other documents show evidence of particular focus on gender issues? Project document review 4.3 What percentage of participants trained under SEDA are female? Project document review 4.4 What is the gender breakdown of CDCs receiving assistance under SEDA? Project document review 4.5 To what extent did female community members play an active role in the design and implementation of SEGA infrastructure projects? Project document review Key informant interviews Focus group discussions Program narratives Key informant interview protocol Focus group discussion protocol Community fora protocol Evaluation Question Data Sources Instruments Notes 4.6 Are any participating CSOs focused on women’s issues? Project document review Key informant interviews Focus group discussions Program narratives Key informant interview protocol Focus group discussion protocol Community fora protocol Network mapping protocol 4.7 What percentage of SEDA program staff are female? Project document review 4.8 Do any program narratives or success stories focus on women’s issues? Program narratives 4.9 Do SEDA publications on citizen participation, Public Councils and PPPs address gender issues? Project document review Key informant interviews Key informant interview protocol 4.10 Do female beneficiaries report an improvement in their local communities as a result of SEDA? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol Community fora protocol 4.11 How does this compare with males’? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol Community fora protocol 4.12 Do female community members believe that SEDA offered them an opportunity to play meaningful roles in their communities? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol Community fora protocol 4.13 Did SEDA facilitate the development of female leadership in target communities? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol Community fora protocol 4.14 What do women see as the primary challenges to their Key informant interviews Focus group discussions Key informant interview protocol Evaluation Question Data Sources Instruments Notes engagement in community development? Focus group discussion protocol Community fora protocol 4.15 Do women program beneficiaries feel empowered as a result of the project? Key informant interviews Focus group discussions Key informant interview protocol Focus group discussion protocol Community fora protocol 3.0 Methodology, Data Collection and Targets The aim of the data collection for the SEDA performance evaluation is to systematically collect the information required to identify findings through answers to the evaluation questions. The data collection phase will include coordination with the SEDA teams, contacts with stakeholders and counterparts, and the application of data collection methods. The main data collection methods to be applied, within this phase, are the interviews with stakeholders, focus group discussions, key informant survey, survey for wider network of CSOs, donors and the follow up interviews with the SEDA project teams in Baku and in the regions. In this phase, the triangulation strategy based on the interviews will serve to question and/or verify the preliminary findings from the desk phase. In the interviews with the different stakeholders, their assessment of the project will be sought. While the individual assessments may be biased in various ways, triangulation will reveal a pattern of interest to the overall assessment of the evaluators. Field visits and meetings with partners in target communities will be an opportunity to gain input from some of the target beneficiaries, to conduct unstructured field observation, and gather best practices and lessons learned from program implementation as well to observe changes towards achievement of the SEDA objectives. Key Informant Surveys. In the event that the evaluation team (ET) is not able to interview a particular constituency or key actor, it will identify alternate sources as proxies who are able to provide similar information. These interviews will be conducted in person whenever possible, although in a few cases the Team may be limited to telephone or e-mail communications. Therefore, KII data from interviews will be supplemented by a limited number of key informant surveys utilizing a questionnaire with scaled items from the interview protocols. Key informant surveys will be employed when unique key informants are unavailable for in-person interviews but willing to complete a written survey on-line, or if a group of key informants in a single site or institution is willing to complete the written questionnaire format. By utilizing these standard questions across evaluation instruments and target populations we increase consistency and uniformity of data and strengthen the validity of the findings. Online survey for CSOs. An online survey will be developed as an important tool to complement narrative data collected during field visits and interviews. It will be carefully designed in order to allow respondents to provide information, share views and opinions while remaining anonymous. One questionnaire for CSOs participating in trainings and other SEDA related active ties will be developed for the purpose of this evaluation. The purpose of the surveys for this evaluation is to reach out to a majority of CSO partners and beneficiaries of the SEDA project, particularly those with whom the ET will not meet personally during the limited field phase. As internet access is apparently high among CSOs that participated in the project, the survey will be conducted using a web-based tool such as Survey Monkey™, to ensure a confidential, easily accessible, and fast methodology of collecting and analyzing data. That tool will be supplemented by telephone and email follow￾ups to boost response rates, which may initially be low. Response rates will also depend greatly on whether contact information available from EWI is up to date. Emailing of the questions by the automated online system will be preceded by an emailed introduction to the survey and explanation of its purposes, together with assurances of confidentiality of responses. The survey instrument will first be developed in English for approval by the Mission, before being translated. Survey questions will have closed responses, thus obviating the need for translating them into English, and facilitating its completion by respondents. Focus Group discussions. A minimum of one focus group discussion in each local community will be organized in order to allow participants to share opinions and ideas in the course of a discussion on a specified theme. Focus group discussion guides have been developed for each of the target groups, and will assist systematic analysis of the data collected. The ET team will conduct focus group discussions with CSOs and beneficiaries of the project in Baku, Aran, Quba-Khachmaz, and Ganja-Qazakh. Protocols for the FGDswill consist of a limited number of standard stimulus items aimed at establishing rapport and ensuring broad input, as well as of a short series of probes aimed at eliciting input and gauging differing opinions and perspectives. During FGDs, the facilitator will keep the discussion moving as required, remind participants that all ideas and opinions are considered valid input, and prevent any single group or individual from dominating the session. These facilitated discussions with 5-8 participants each will allow for exchange and discussion of experiences and viewpoints, and enable the ET to probe for context analysis, achievements of the projects, lessons learned and recommendations for future phase of project implementation. This information will serve to complement the data collected through the survey across the country. Focus group discussions will take a maximum of 1.5 hours. 3.1 Sampling Through the field phase that includes interviews, focus group discussions, mini survey, and site observations the ET will reach out to at least 104 representatives of beneficiaries, government counterparts, donors and CSOs. However, it should be noted that compared to the number of individuals who have participated and benefited from SEDA interventions during the implementation period, the Assessment sample size is not large enough to statistically generalize the findings to the rest of the beneficiary population. The sample does, however, offer a general pattern of opinions and perspectives that, when triangulated, offer evaluative “impressions.” Individuals and organizations to be included in the sample will be limited by circumstances on the ground. Geographic coverage is essential to this performance evaluation. Because SEDA is a project based in three diverse regional areas of Azerbaijan, each with several implementation sites, the ET will be traveling to all three regions - Aran, Quba-Khachmaz, and Ganja-Qazakh - to interview recipients of the SEDA project’s interventions. Due to time limitations and logistical requirements, the specific sites for data collection were purposively selected to provide array of contexts in which SEDA is being implemented generally representative of the total population of sites. As presented in the table below, the specific Rayons13/towns to be visited in the regions are: Imishli, Beylagan and Sabirabad in Aran; Shabran, Quba and Khachmaz in Quba-Khachmaz; and Ganja, Goranboy and Samukh in Ganja-Qazakh). The ET 13Rayons are administrative districts within Regions of Azerbaijan. will organize interviews and focus group discussions (FGDs) in the communities visited and also in Baku (if applicable). While these sample locations do not offer input from every participating Rayon or town, they should enable the Evaluation Team to glean a sufficient understanding, when combined with data collected from interviews, site observations as well as questionnaires implemented in areas not physically visited, to present valid conclusions and relevant recommendations. Selection of people for interviews will be done in coordination between ET and SEDA Team, based on non-probability sampling without resorting to random selection due to time and resources constrains. Particular attention will be paid to interviews with women who participated in the activities in order to gauge the impact of program activities on them and to better understand the gender dynamics of program implementation and outcomes. Number of Site Visits in Three Target Regional Areas of Azerbaijan Aran Ganja-Qazakh Quba-Khachmaz Rayons Site Type Communities to be visited Rayons Site Type Communities to be visited Rayons Site Type Communities to be visited Imishli Rural 2 Ganja Urban 1 Quba Town 4 Sabirabad Rural 4 Samukh Rural 2 Khachmaz Town 1 Beylagan Rural 2 Barda Rural X Shabran Rural 1 Shirvan Rural X Goranboy Rural 1 Mingechevir Urban X Shamkir Town X Qabala Town X Data Collection Activity – Data Collection Methodology Data Source: Notes Date(s) Main parties (N=2): • USAID • EWMI Key partners of SEDA (N=5): • UMID Social Support • ICNL Brief-in with USAID and EWMI. (N=2) Individual Initial interviews Individual Initial Design Feedback Discussion, interview (N=7) Report Review & Feedback (N=7) Key partners will have individual interviews and also provide on￾going input into design and report. 30 November – 5 December 2015 14 December 2015 Activity – Data Collection Methodology Data Source: Notes Date(s) • Constitutional Research Fund • Ministry of Economy and Industry • Council on State Support to NGOs interviews and meetings (N=5) CSOs in Baku (N=3) • Social Progress Public Union • National Resource Training Centre National Resource Training Centers (N=5) • Quba • Shamkir • Shirvan • Mingachevir • Qabala Individual Interviews (N=3) Online survey Interview (N=3 members) - Interview feedback (N=3) Online survey feedback (N=5) CSO representatives in Baku will be interviewed. Online survey will be sent to all National Resource Centers 30 November – 5 December 2015 14 December 2015 30 November – 5 December 2015 14 December 2015 SEDA offices (N=3) • SEDA IMISHLI office • SEDA GUBA￾KHACHMAZ Office • SEDA GANJA￾GAZAKH office Individual interviews and meetings (N=3) Focus Group Discusssion (FGD) Interview feedback (3) Interview Feedback (3) Individual interviews will be conducted with SEDA offices 7 December – 12 December 2015 Activity – Data Collection Methodology Data Source: Notes Date(s) Site observation Meetings in communities in Aran (N=8) • Imishli (meetings in 2 communities) • Sabirabad (meetings in 4 communities) • Beylagan (meetings in 2 communities) Communities: Mamishlar, Imishli, Ashaghi Chemenli, Jahar, Zangana, Sabidabad, Beylagan, Sabirabad. Individual interviews and mtgs. (N=8) Focus Group Discusssion (FGD) Site observation Focus Group Discusssions (N= 8 members) Feedback from interviews and focus group discussions Focus Group Discusssions will be held with CDCs, beneficiaries, NGOs and municipalities in 8 communities in Aran. 7 December – 8 December 2015 SEDA Civic Grant Recipients (N=7) Online survey Online survey feedback (N=7) Feedback from online survey (N=7) - 30 November – 5 December 2015 Meetings in communities in GUBA-Khachmaz region (N=6) • Quba (meetings in 4 communities) • Khachmaz (meeting in Meetings. Focus Group Discusssion (FGD) Site observation 6 Focus Group Discusssions Feedback from group discussions and meetings (N=6) Focus Group Discusssions will be held with CDCs, NGOs, beneficiaries and municipalities in 6 9 December - 11 December 2015 Activity – Data Collection Methodology Data Source: Notes Date(s) 1 community) • Shabran (meeting in 1 community) Communities: Quba, Amirkhanli, Pustaqasim, Aghayazi Buduq, Alpan, Rustov communities in Quba-Khachmaz region. Meetings in communities in Ganja-Qazakh region (N=4) • Ganja (meeting in 1 community) • Samukh (meetings in 2 communities) • Goranboy (meeting in 1 community) Communities: Ganja, Ali Bayramli, Qushchular, Seyidler, Individual interviews and mtgs. Focus Group Discusssion (FGD) Site observation Focus Group Discusssions Feedback group discussions (N=4) Group meetings will be held with CDCs, NGOs, beneficiaries and municipalities in 4 communities in Ganja-Qazakh region. 9 December - 11 December 2015 3.2 Methodology Limitations There are several limitations inherent to the design of this evaluation: 4. In light of the time constraints on data collection and analysis, the focus on a limited number of relevant key informants and field-work conducted across potentially 50+ communities in three regions, random sampling is not an appropriate or realistic option. While targeted purposive and opportunistic sampling will generate sufficient data to reliably address the four key evaluation questions, findings cannot be generalized to the population as a whole (sampling bias). 5. As some key informants may decline to be interviewed, there is a possibility of selection bias, i.e. those respondents who choose to be interviewed might differ from those who do not in terms of their attitudes and perceptions, affiliation with government/non-government structures, and socio-demographic characteristics and experience. This may apply to in-person interviews and mini-survey as well. 6. Since a number of questions during the interviews will deal with issues that took place in the past, recall bias cannot be excluded. As SEDA activities were launched in 2011, some respondents may find it difficult to accurately compare organizational arrangements/access to services three years from prior to now. 7. There is a general tendency among respondents to under-report socially undesirable answers and alter their responses to approximate what they perceive as the social norm (halo bias). The extent to which respondents will be prepared to reveal their true opinions may also vary for some questions that call upon the respondents to assess the performance of their colleagues or people on whom they depend upon for the provision of services. To mitigate this limitation, ME&A will: provide the respondents with confidentiality and anonymity guarantees, where possible; conduct the interviews in the settings where respondents feel comfortable; and establish rapport between the interviewer and the respondent. 8. Other possible limitation may include low response rates and disproportionate regional participation in one-on-one interviews. If this is the case, the results will represent opinions of the most active and open respondents whose experience with SEDA is likely to differ from that of those who decline to be interviewed. To avoid this outcome, full cooperation of SEDA staff and national counterparts in arranging interview process is required. 9. Cultural/religious sensitivities concerning the propriety of women’s participation in male-led interviews or FGDs could serve to limit female participation in the evaluation. Female evaluation team members will therefore take the lead in recruiting women respondents and serve as lead interviewers and facilitators in such circumstances. The potentially distortive effects of these methodological limitations and possible biases will be mitigated through the triangulation of inputs and evidence addressing each question and careful examination of results to identify inconsistencies and evidence of social desirability. 4.0 Evaluation Schedule The evaluation schedule is presented below. Azerbaijan SEDA Evaluation Schedule ◄October 2015 ~ November 2015 – January 2016 ~ February 2016► Sun Mon Tue Wed Thu Fri Sat Nov 15 16 Conference call w/USAID Review Materials Home 17 Review Materials Home 18 Review Materials Home 19 Review Materials Mtgs & Interviews Washington, DC 20 Submit Work Plan/Eval Design Mtgs & Interviews Washington, DC 21 22 23 24 25 26 US: Thanksgiving 27 Travel to Azerbaijan 28 Arrive in Azerbaijan 29 Baku 30 In-Briefing w/USAID Team Planning Meeting Baku Dec 1 Receive USAID Comments on WP KIIs Baku 2 Submit Final Work Plan/Eval Design KIIs Baku 3 Key Informant Interviews Baku 4 Key Informant Interviews Baku 5 Write-up Notes Baku 6 Baku 7 Mtgs with communities, including Imishli, Mamishlar, Sabirabad, Ashaghi Chemenli. Aran 8 Mtgs with communities including Beylegan, Jahar, Ahmadabad, Zangana. Aran 9 Mtgs with communities including Quba, Amirkhanli, Ganja, Ali Bayramli. T1: Guba Khachamaz T2: Ganja￾Gazakh 10 Mtgs with communities including Pustaqasim, Aghayazi Buduq, Qushchular and Seyidler. T1: Guba Khachamaz T2: Ganja￾Gazakh 11 Mtgs with communities including Alpan, Rustov, Alibayramli. T1: Guba Khachamaz T2: Ganja￾Gazakh 12 Baku 13 Baku 14 Mtgs & Interviews with Stakeholders Baku 15 Interim briefing w/USAID Final interviews Baku 16 Write-up Notes, Outline of Main Findings & Recs Baku 17 Write-up Notes, Develop Outline of Main Findings & Recs Baku 18 De-briefing w/USAID Baku 19 Travel Home 20 21 Write Draft Report Home 22 Write Draft Report Home 23 24 25 Christmas 26 27 28 Write Draft Report Home 29 Write Draft Report Home 30 31 New Years’ Eve Jan 1 New Years’ Day 2 Azerbaijan SEDA Evaluation Schedule ◄October 2015 ~ November 2015 – January 2016 ~ February 2016► Sun Mon Tue Wed Thu Fri Sat 3 4 Write Draft Report Home 5 Write Draft Report Home 6 Write Draft Report Home 7 Write Draft Report Home 8 Submit Draft Report Home 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Receive USAID Comments 26 Integrate Comments Home 27 Integrate Comments Home 28 Integrate Comments Home 29 Integrate Comments Home 30 31 Feb 1 Submit Final Report Home 2 3 4 5 6 ANNEX 3. EVALUATION INSTRUMENTS SEDA Performance Evaluation Key Informant Interview (KII) Protocol Evaluation questions, as set by the Terms of Reference for the SEDA Evaluation are the following: • How effective has the implementation of each component been? • How complimentary were the activities under the three components identified in the scope of the project towards each other? • Which interventions were the most effective towards achieving complementarity of results? • How sustainable are the established dialogue and relationships between CDCs, other civil society and government officials? • How successful was SEDA project in integrating gender considerations? The following sections present the Interview guides developed for data collection towards responding to the Evaluation questions as set out for the Evaluation of the SEDA Project. General methodological notes: Each interview, discussion group will start with the presentation of the Evaluation team and of the evaluation objectives, followed by the presentation of the interlocutors. Whenever necessary, a brief presentation of the Project will be also done. The questions will be sent in advance to the people who are going to be interviewed. Interviews will last approximately 1-1.5 hours each. The participants in discussion groups will be briefed in advance about the major topics to be discussed during the meeting. The discussion groups will be composed of 6-12 people, depending on the context of the community/thematic area. The discussion groups will last 1.5 hours each and will take place during the site visits to the sampled municipalities. In line with best evaluation practices, the interviews and focus groups and discussion groups will be attended only by the evaluator/s, the interviewed people and the interpreter, if the case. Interview Guide for SEDA management and project staff 1. What are the major achievements of the Project (within your respective component) to date that you are proud of? What was the most challenging in achieving these results? Are there any planned results which could not be achieved until the end of the Project (within your respective component) and why? 2. To what extent did women participate in the planning and implementation of the activities under each component? What are the main challenges in involving women? 3. Which capacity building activities and mechanisms used up until now were the most / least successful and why? What was the impact of training delivered by the Project so far on [depending on component]: a) government staff and b) community leaders (members of CDC); c) other community members (if/where applicable)? 4. Did some communities perform better than others and why? 5. What are the most tangible benefits of project implementation so far at local/rayon/central level (within your respective component)? 6. To what extent do [local/regional/central] authorities appear to understand/support the meaningful role of CDCs/other forms of citizen participation at the local/regional/central levels? 7. In general, how satisfied are you with the overall project management and implementation? In particular timelines, efficiency in utilization of financial and human resources, coordination with others? 8. How would you describe SEDA’s cooperation with the central/rayon/local authorities? What went well? What could have been done better? 9. What difference has SEDA made via this Project for community groups in terms of enabling citizens to have a greater voice in determining local development needs and priorities? How about women in particular? 10. What is in your view the value added of the Project? 11. How is the component you focus on relating to other components? What synergies are happening? 12. Looking ahead, which of the achievements to date are likely to be sustained or expanded without further external support? Which of them will require further support? What is the likeliness that the communities will utilise the capacities of CDCs and networks and increase community participation in strategic planning/prioritisation? 13. In your opinion, which are the top three priorities of the socio-economic/civic participation that needs to be addressed in the coming years? Do you see any particular role of USAID or SEDA (in particular) in addressing these needs? Thank you again for your participation in this interview. Your input is a valuable resource for our assessment and we appreciate your willingness to take the time to share your thoughts and opinions. If you have any additional thoughts or questions about the interview or the evaluation study, please feel free to contact me at (give e-mail address). If you wish, we would also be pleased to send you a summary of the evaluation findings and recommendations once it is approved for public use. Interview Guide for government counterparts (central/local levels – as per component and overall) 1. To what extent is the Project aligned with country’s needs in the area of socio-economic development/civic participation? Were some needs better addressed in the Project implementation than others and why? 2. Are you aware of any outstanding results achieved by the Project so far? Who has benefited most from the Project? If your organisation/institution has benefited, please describe how. 3. In your knowledge, to what extent did women participate in the planning and implementation of the activities under each component? What are the main challenges in involving women? 4. In your opinion, are these achievements likely to be sustainable? Please motivate your answer. 5. Which capacity building activities and mechanisms were the most / least successful in enabling your institution to improve/establish mechanisms/legislation? 6. To what extent has the Project contributed to date to increasing the number of citizens participating in community prioritisation (strategic planning)? What are the systemic obstacles impeding the citizen participation? What about women? 7. In your opinion, which are the top three priorities of the local development that need to be addressed in the coming years? Do you see any particular role of USAID (or SEDA in particular) and international donors in addressing these needs? Thank you again for your participation in this interview. Your input is a valuable resource for our assessment and we appreciate your willingness to take the time to share your thoughts and opinions. If you have any additional thoughts or questions about the interview or the evaluation study, please feel free to contact me at (give e-mail address). If you wish, we would also be pleased to send you a summary of the evaluation findings and recommendations once it is approved for public use. Interview Guide for international donors 1. How does the Project align with Azerbaijan’s needs and international commitments? 2. In your view what were the major achievements of the Project to date and at which levels was it most / least successful? In your view, which were the factors that enabled or hampered the attainment of project results so far? 3. Have you noticed any significant drawbacks and what worked well in the implementation? How did the project ensure co-ordination with other similar interventions funded by your organization and other donors to encourage synergy and avoid overlaps? 4. Where was the most significant impact/difference made by the Project? To what extent did women benefit/were empowered through activities under each component? What are the main challenges in involving women in developmental processes in Azerbaijan? 5. Would you consider the results of the Project to be likely sustainable or do you believe additional donor interventions/projects are necessary to maintain the achieved levels of capacities and mechanisms? Which are the top three priorities of the local development and citizen participation that needs to be addressed in the coming years? Which of them is [interviewed donor agency] planning to address in the future and how? Thank you again for your participation in this interview. Your input is a valuable resource for our assessment and we appreciate your willingness to take the time to share your thoughts and opinions. If you have any additional thoughts or questions about the interview or the evaluation study, please feel free to contact me at (give e-mail address). If you wish, we would also be pleased to send you a summary of the evaluation findings and recommendations once it is approved for public use. Guide for Discussion Groups with local stakeholders (representatives of local governments, CDC) Introduction • Introduction of the evaluator/s to the group • Presentation of participants • Provision of background information to the discussion group: • The purpose of the discussion • The intended recipients of findings and how they will be used • How feedback will be handled (issues of anonymity, confidentiality, data protection) • Rules of the discussion group: who speaks when and agreement on how to indicate when one wants to speak • The time allocated for discussion and explanation of the discussion group approach • Answering any questions participants might have. Discussion • How do you see your role in the process of community development council development and work on prioritisation of local projects? • How did the Project support your efforts? • What do you think have been the biggest achievements of the Project in your community? Would it have been possible to achieve these changes (if any) without the Project? • To what extent did women participate in the planning and implementation of the activities in communities/CDC? What have been the main benefits for women? • What challenges do you face? What are the main challenges in involving women? • Has your community ensured support/funds for continuation of practices initiated by the project? If yes, in what way? If no, why not? • What are the challenges ahead and ways to overcome them? What would your role be in this process? • What are the three priorities of the local development [and citizen participation] that needs to be addressed in the coming years? End of Discussion Thanking participants for attending and giving feedback Guide for discussion Groups with training participants Introduction • Introduction of the evaluator/s to the group • Presentation of participants • Provision of background information to the discussion group: • The purpose of the discussion • The intended recipients of findings and how they will be used • How feedback will be handled (issues of anonymity, confidentiality, data protection) • Rules of the discussion group: who speaks when and agreement on how to indicate when one wants to speak • The time allocated for discussion and explanation of the discussion group approach • Answering any questions participants might have. Discussion 1. Exploring actual learning experience • Which new skills and knowledge do you retain following the trainings and technical assistance delivered by the Project? Do you apply new skills and approaches in your daily routine? • To what extent did women participate in the training/capacity building activities? What have been the main benefits for women? • Are there further needs for capacity building? In which area? 2. Exploring actual changes in behaviours and attitudes • What is the profile of work you are engaged with with? • How do you see your role in the process of improving citizen participation/socio-economic development/local strategic planning/other? • What do you think have been the biggest achievements of the Project to date in terms of citizen participation? • Would have it been possible to achieve these changes (if any) without the project? • What are the challenges ahead and ways to overcome them? Do you see any particular role for you in this process? End of Discussion • Thanking participants for attending and giving feedback. Guide for discussion Groups (women, men, young people beneficiaries of supported projects) Introduction • Introduction of the evaluator/s to the group • Presentation of participants • Provision of background information to the discussion group: - The purpose of the discussion - The intended recipients of findings and how they will be used - How feedback will be handled (issues of anonymity, confidentiality, data protection) - Rules of the discussion group: who speaks when and agreement on how to indicate when one wants to speak - The time allocated for discussion and explanation of the discussion group approach • Answering any questions participants might have. Discussion • How did you learn about the (new) activity/initiative [as per initiative organised by project] available in your municipality? • What motivated you to take part in the initiative? • What was the most tangible benefit to you and your family life that you would highlight as a result of the project? • To what extent did women participate in these activities? What have been the main benefits for women? • What challenges do you face? What are the main challenges in involving women? • What priority needs do you still have and how could be addressed in the future? End of Discussion Thanking participants for attending and giving feedback. Focus Group Discussion Protocol/Prompts Focus Group Discussion Guide Local Beneficiaries/Community Members Introduction: Since October 2011 the Socio-Economic Development Activity (SEDA) Project has been helping local communities and local authorities work together to identify and address social and economic development needs of their communities. To help build local capacity, SEDA has sponsored training courses for Community Development Councils (CDCs), helped facilitate local infrastructure projects addressing identified community needs, and provided Civic Engagement Grants to local NGOs to enable them to pursue community advocacy. SEDA is being conducted by the East-West Management Institute (EWMI) and its local partners and is sponsored by the United States Agency for International Development (USAID) and the Government of Azerbaijan. Our team from ME&A - a U.S. international development organization – is conducting an evaluation of SEDA to identify how well it is working and recommend any areas for improvement. Each of you live in a city, town or village where SEDA has worked and may have participated in a SEDA sponsored training or activity. The aim of this focus group discussion is to enable us to learn as much as we can about your experiences and views. Ground Rules: First, here are a few “ground rules” to help produce a productive discussion: 1. Only one person should speak at a time; 2. Please no side conversations with those sitting near you; 3. Let’savoidhavingoneortwopeopledominatetheconversation;andweneedto 4. Be sure to hear from everyone; we want to hear as many different voices, stories and perspectives as possible. Opening Prompt: To get started, we will go around the room to have every one briefly respond to the following question: What one key fact do you want us to know about your local community? Follow On Prompts: 1. How would you characterize the relationship between your community and the local government authorities; does it work for the betterment of the community? 2. Is there a CDC in your community and have you participated in any of its meetings, training or other events? 3. Have you heard about the SEDA project or been involved in any SEDA-sponsored events? 4. In your opinion has the social and economic situation in your community changed at all in the past four years? 5. In your opinion, what has been the role of SEDA activity, if any, in these changes? 6. Do you feel that it is possible for you to play a meaningful role in community social and economic development? 7. If you look ahead to the year 2020, what are your hopes and expectations for your local community? Concluding Statement: Thank you for participating in this focus group discussion. Your contributions have been quite helpful. Should you find that you have other inputs to share or other comments or suggestions please contact us at xxxx xxxxxx (phone number) QUESTIONNAIRE FOR CDC MEMBERS Introduction: SinceOctober2011the Socio-Economic Development Activity (SEDA) Project has been helping local communities and local authorities work together to identify and address social and economic development needs of their communities. To help build local capacity, SEDA has sponsored training courses for Community Development Councils (CDCs), helped facilitate local infrastructure projects addressing identified community needs. Each of you live in a city, town or village where SEDA has worked and may have participated in a SEDA sponsored training or activity. The aim of this short questionnaire is to enable us to learn about your experiences and views on this Project. 1. Overall, the SEDA project seeks to help improve the ways local community members, community organizations and local government officials work together to create socio￾economic development and address key needs and priorities in their villages and regions. In your view, how did the Project perform on this in Azerbaijan? Extremely Poorly1_____2_____3_____4_____5_____Extremely Well 2. On a five- point scale in which one represents Not effective at all and five, Extremely effective, please rate the effectiveness of the training courses in building the capacity of your CDC: Not effective at all1_____2_____3_____4_____5_____ Extremely effective 3. On a five point scale in which one represents extremely closed and five, extremely open, how would you rate the openness of the process for development of infrastructure project to the involvement and input of all interested community members: Extremely Closed1____2_____3_____4_____5_____Extremely Open 4. How would you rate the overall support of the SEDA Project to CDC? Not useful at all1_____2_____3_____4_____5_____ Extremely useful 5. In your opinion how likely is it that the community will develop and implement other such projects in future? Extremely Unlikely1____2_____3_____4_____5_____Extremely Likely 6. Will you remain participating in such forum (e.g. CDC or Public Council) in the future?  Yes  No  Maybe  I don’t know Thank you! QUESTIONNAIRE FOR CSOs Introduction: SinceOctober2011,Socio-Economic Development Activity (SEDA) Project has been helping local communities and local authorities work together to identify and address social and economic development needs of their communities. To help build local capacity, SEDA has sponsored training courses for Community Development Councils (CDCs), helped facilitate local infrastructure projects addressing identified community needs. Each of you live in a city, town or village where SEDA has worked and may have participated in a SEDA sponsored training or activity. The aim of this short questionnaire is to enable us to learn about your experiences and views on this Project. 1. Overall, the SEDA project seeks to help improve the ways local community members, community organizations and local government officials work together to create socio￾economic development and address key needs and priorities in their villages and regions. In your view, how did the Project perform on this in Azerbaijan? Extremely Poorly1_____2_____3_____4_____5_____Extremely Well 2. On a five- point scale in which one represents Not effective at all and five, Extremely effective, please rate the effectiveness of the training courses in building the capacity of your CSO: Not effective at all1_____2_____3_____4_____5_____ Extremely effective 3. How would you rate the effectiveness of the SEDA project in building the capacity of local NGOs/CSOs overall? Not effective at all1_____2_____3_____4_____5_____ Extremely effective 4. How effective are the Regional NGO Resource and Training Centers in strengthening NGOs and other local civil society organizations? Not effective at all1_____2_____3_____4_____5_____ Extremely effective 5. How would you rate the overall support of the SEDA Project to NGOs? Not useful at all1_____2_____3_____4_____5_____ Extremely useful Thank you! QUESTIONNAIRE FOR BENEFICIARIES Introduction: The Socio-Economic Development Activity (SEDA) Project has been helping local communities and local authorities work together to identify and address social and economic development needs of their communities. To help build local capacity, SEDA has sponsored training courses for Community Development Councils (CDCs), helped facilitate local infrastructure projects addressing identified community needs. Each of you live in a city, town or village where SEDA has worked and may have participated in a SEDA sponsored training or activity. The aim of this short questionnaire is to enable us to learn about your experiences and views on this Project. 1. Overall, the SEDA project seeks to help improve the ways local community members, community organizations and local government officials work together to create socio￾economic development and address key needs and priorities in their villages and regions. In your view, how did the Project perform on this in your community? Extremely Poorly1_____2_____3_____4_____5_____Extremely Well 2. On a five point scale in which one represents extremely closed and five, extremely open, how would you rate the openness of the infrastructure project to the involvement and input of all interested community members: Extremely Closed1____2_____3_____4_____5_____Extremely Open 3. How important has the infrastructure project been (or you expect it to be) to the well-being of your village: Not at all important1____2_____3_____4_____5_____Extremely Important 4. In your opinion how likely is it that the community will develop and implement other such projects in future? Extremely Unlikely1____2_____3_____4_____5_____Extremely Likely Thank you! Community Service Organization (CSO) Survey (via Survey Monkey) Performance Evaluation of the Socio-Economic Development Activity (SEDA) Introduction: We are writing to enlist your participation in a brief (ten minutes or less) on-line survey about the SEDA project. SinceOctober2011the Socio-Economic Development Activity (SEDA) Project has been helping local communities and local authorities work together to identify and address social and economic development needs of their communities. To help build local capacity, SEDA has sponsored training courses for Community Development Councils (CDCs), helped facilitate local infrastructure projects addressing identified community needs, and provided Civic Engagement Grants to local NGOs to enable them to pursue community advocacy. SEDA is being implemented by the East-West Management Institute (EWMI) and its local partners and is sponsored by the United States Agency for International Development (USAID) and the Government of Azerbaijan. Our team from ME&A - a U.S. international development organization – is conducting an evaluation of SEDA to identify how well it is working and recommend any areas for improvement. It is our understanding that you are affiliated with a Community Services Organization (CSO) that has participated in the SEDA project. Your views on this project will therefore be extremely helpful to us in conducting the SEDA evaluation. Your participation is entirely voluntary and there will be no consequences to you or your CSO if you decline to participate. The survey is entirely anonymous and your name will not be included in any evaluation report. Your participation in this survey is greatly appreciated. I you have any questions or suggestions please contact the SEDA evaluation team at: zkacapor@gmail.com Online Survey for CSO representatives 1. Your role in the organization: a. Director b. Administrative/finance staff member c. Supervisor d. Staff Member e. Volunteer f. Other, please elaborate: 2. What are the main areas of your work or focus of your organization? Please, mark all that apply: a. Advocacy and monitoring of policies b. Protection of women’s rights c. Socio-economic development d. Education e. Health f. Youth Services g. Community engagement h. Sports and recreation i. Arts j. Other, please describe: ____________ 3. What external factors influence the work of your CSO? Please check all that apply • Political situation in the country • Socio-economic factors and poverty • Regional isolation • Competitiveness with other CSOs • Governmental opposition or indifference • Tradition of non-participation in local governance • Other_____ 4. Does/did your organization receive a grant or other support from SEDA? o Yes o No 5. If yes, which kind? a. Community Self-Help Grants program b. SEDA Civic Engagement Grants c. Community Socio-Economic Projects d. Other e. Not sure 6. How would you rate your experience with the grant(s) you have received from SEDA? a. Excellent b. Good c. Average d. Poor e. Terrible 7. What were the three main achievement of your project? Please list below: i. ii. iii. 8. In which SEDA-sponsored trainings did you participate? a. SEDA’s CSO Advocacy Training Program b. Organizational Capacity Development Workshops c. Conducting meaningful meetings d. Decision making e. Needs assessment f. Stakeholder analysis g. Project planning, design and implementation h. Environmental impact i. Cost-benefit analysis j. Vocational training k. Support projects in animal husbandry and veterinary care l. Modern crop production practices m. Grape production, n. General agriculture, o. Rural and environmental development p. Other, please list q. None 9. How would you assess the trainings you received from SEDA? a. Very useful b. Useful c. Average d. Not very useful e. Not at all useful 10. What was the most beneficial thing you learned in the trainings? ______________ 11. Did you participate in study tours? • Yes • No If yes, how would you rate it? a. Very useful b. Useful c. Average d. Not very useful e. Not at all useful 12. Did you take part in learning circle for CSOs? • Yes • No If yes, how would you rate it? a. Very useful b. Useful c. Average d. Not very useful e. Not at all useful 13. Please assess the changes in your organization as a result of the capacity building support you have received from SEDA Project: Question Yes, definitely Yes, a little No change No, it is worse We are more focused on what we want to achieve We have a better strategy We have more professional expertise in our organization thanks to trainings we attended Advocacy initiatives are now better designed We understand power dynamics and power analysis better (for advocacy) We have skills to conduct better context analysis and research on a topic of interest We have more funding sources We have better relationship with the authorities We have better cooperation with other organizations that participated in the trainings We are better known and respected in our community We increased coverage of gender issues We have improved our monitoring and evaluation of advocacy capacity 14. Overall, how would you assess your cooperation with SEDA? o Excellent o Good o Average o Poor o Terrible 15. In your opinion, to what extent has the SESA Project contributed to increasing local civic participation and collaboration between civil society and local government? i. A tremendous amount ii. Quite a bit iii. Somewhat iv. Slightly v. Not at all ______________________ Other comments Annex 4. Evaluation Schedule Wee k Sunda y Monday Tuesday Wednesday Thursday Friday Saturday Week 1 Arrival 1.30 pm – In￾briefing with USAID (all) 4 pm – SEDA Team mtg (all) 9 am – Team meeting (all) 10 am – UMID (all) 12 pm – MG Consulting (all) 2 pm – Team meeting, logistics (all) 10 am – Council for NGOs (JS, ZKD) 10 am – 1 pm – SEDA mtg re. logistics (BB) 11 am – USAID D&G (JS, ZKD) 1 pm – 5 pm – SEDA Component leaders interviews (all) Team meeting – reflection week 1 (all) Week 2 all JS/B B ZKD/Ergi n JS/B B ZKD/Ergi n JS/B B ZKD/Ergi n JS/B B ZKD/Ergi n JS/B B ZKD/Ergi n JS/B B ZKD/Ergi n Week 3 Am/pm – Outbriefing USAID (all) Annex 5. List of Interviewees Category of informant: interviews Institution/organization Informant name and Position Key Informant Interviews USAID Parviz Musayev, Programme manager EWMI/SEDA Timothy Madigan, Chief of Party Vagif Hasanov, Deputy Chief of Party Oleg Amrahov, SEDA Project and Grant Manager Ahmad Shirinov, Economic Development Advisor Katherine Lauffer, Advisor ICNL/MG Consulting Mahammad Quluzade, Chairman ‘Social Progress’ Public Union Qalib Nabiyev, Chairman ‘Umid’ Support to Social Development Public Union Israyil Isgandarov, Executive Director Rasim Quliyev, Program Director The Council on State Support to NGOs under the auspices of the President of the Republic of Azerbaijan Farasat Qurbanov, Executive Director Ministry of Economy and Industry Elshan Asadov, Head of the Department of Regional Development and State Programs Javid Qadimov, Senior Advisor of the Department of Regional Development and State Programs Toghrul Guliyev, Deputy Director, Department on Cooperation with International Organizations Ruslan Rustamli, Representative of the Department on Cooperation with International Organizations Beylaqan Youth Enlightenment and Zaur Quliyev, Chairman Social and Development Public Union Azerbaijan National NGO Forum/ Baku NGO Resource and Training Center Rauf Zeyni, President World Bank AZRIP Project Anar Azimov, Monitoring and Evaluation Officer ‘Constitution’ Researches Foundation Alimammad Nuriyev, President Ministry of Education in Quba Loghman Muslumov, Director SEDA offices in targeted regions SEDA office in Imishli (Aran) Ahmad Mammadzada, Regional Director Jabir Alizada, Advocacy Coordinator Kamala Azizaliyeva, Administrative Assistant Emin Kazimov, Program Development Specialist Bahar Miriyeva, Community Specialist SEDA office in Ganja (Ganja-Qazakh) Gunay Zeynalova, Regional Director Gulayat Huseynova, Community Specialist Farkhad Huseynov, Community Specialist Zohrab Zohrabov, Program Development Specialist Aynur Ismayilova, Advocacy Coordinator Aysel Abbasova, Administrative Assistant SEDA office in Quba (Quba￾Khachmaz) Eynulla Kheyrullayev, Regional Director Albina Qasimova, Administrative Assistant Elshan Kerimov, Program Development Specialist Zuleykha Amirova, Community Specialist CSOs based in Ganja-Qazakh region Ganja Euro-Atlantic Information Centre Mehman Khalilov, representative Ganja Regional Advice Centre Public Union Arif Jahangirov, representative Women in Civil Society Public Union Gulnara Hasanova, representative Bridge to the Future Public Union Seymur Valiyeva, representative The Society of Disabled Women Public Union Nigar Talishinski, representative Youth Unit of the Society Disabled Women Natavan Qurbanova, representative ‘Contact’ Regional Development Public Union Sudaba Mammadova, representative CSOs based in Quba -Khachmaz NGO Forum Qorkhmaz Kheyrullayev, representative Quba Regional Resource Centre Anar Murtuzov, representative Aygun Shikhmamayeva, Psychologist ‘Lira’ Young Talents Public Union Lala Abushova, representative Qaranfil Niftaliyeva, representative Regional Media Development Public Union Vali Shikhverdiyev, Chairman Karabakh Veterans and Martyrs Public Union Namiq Gumushlu, representative Reliable Future Public Union Aylan Hakimova, teacher Raskhanim Shabanova, representative Community meetings in Aran Memishler, Sabirabad Nazilagha Quliyev, Chairman of municipality Niyammaddin Mammadov, CDC Coordinator Faraj Hasanov, Chairman of CDC Gulnar Quliyev, Akif Qadirov, Alma Mammadova, Khanzada Yaqubov, Eynarkhan Akbarov, Rovshana Mammadova, Luftiyar Nasirov (community members) Jahar, Imishli Mahammad Ismayilov, Representative of Executive Office Abil Abilov, Mirzali Eyvazov, Mahir Mustafayev, Yengibar Nasirov, Maleyka Isayeva, Lala Ahmadova, Sevda Samadova, Nazifa Javadova (community members) Ashaghi Chemenli, Beylaqan Ifrad Hashimova, Buka Huseynova, Havva Naghiyeva, Khayal Mirzayev, Rahila Jafarova, Mammad Quliyev, Isfandiyar Hashimov, Gulara Alishova, Qalandar Alishov, Mais Huseynov, Mastali Aliyev (community members) Zanqana, Sabirabad Kamran Abuzarov, CDC coordinator Intiqam Alasgarov, Yashar Abbasov, Ruhiyya Suleymanova, Afras Asgarov (community members) Ahmadabad, Sabirabad Isa Allazov, Ismayil Aliyev, Pasha Muradov, Aysha Kocaliyeva, Gulshan Ibrahimova, Jumali Ulfanov, Jorad Jabbarov, Quzqan Mahammadaliyev (community members) Community meetings in Ganja￾Qazakh Ali Bayramli, Samukh Khagani Imanov, Qirqizbay Mammadov, Tahira Abbasova, Qiyafaz Abbasova, Vusala Imanova, Nuru Mustafayev, Sujayat Gozalova, Dunya Berno (community members) Seyidlar, Samukh Fizuli Imanov, Zakir Ashurov, Sahib Abbasov, Mirali Jafarov, Firuz Imanov, Solmaz Kazimova, Nahayat Kazimova, Sahiba Alakbarova, Arifa Abbasova (community members) Qushchular, Goranboy Sakhavat Huseynov, Qabil Abbasov, Hamaya Abdullayeva, Latafat Namazova, Mirvari Naghiyeva, Namik Quliyev (community members) Community meetings in Quba￾Khachmaz Amirkhanli, Shabran Yaver Ibrahimov, Chairman of CDC Rasul Mammadov, Chairman of municipality Hidayat Mammadov, Bahar Huseynov, Jabir Mammadov, Qatifa Ahmadova, Zabita Mammadova (community members) Pustaqasim, Quba Sanan Ilyasov, Chairman of CDC Jamil Hajialiyev, Khayal Allakhveranov, Tural Valiyev (community members) Aghayazi Buduq, Khachmaz Malik Aliyev, Chairman of municipality. Qubad Amirshah, Chairman of CDC. Nizam Pashayev, Shakir Apayev, Beyim Pashayev, Sayyara Qubadova, Pashdi Akhmadov (community members). Alpan, Quba Hikmat Allakhveranov, Natiq Akbarov, Arif Balaqodov, Tural Nazirov, Akif Shahbazov, Qamar Muradova, Elman Dadashov, Qubadov Tofiq (community members) Rustov, Quba Rashad Musayev, Hidayat Alkhazov, Vahid Samadov, Ikram Imamquliyev, Ahmad Taghiyev (community members). Surra, Shabran Akif Mahmudov, Chairman of CDC Annex 6. Site Visits Meetings and Interviews SEDA Performance Evaluation Site Visits, Meetings and Interviews: Field Site Visits Region Rayon Communities Group Meeting Site Observation Aran Sabirabad Memishlar 1 meeting (10 participants: 3 females and 7 males) Yes Zanqana 1 meeting (5 participants: 1 female and 4 males) Yes Ahmadabad 1 meeting (10 participants: 2 females and 8 males) Yes Imishli Jahar 1 meeting (8 participants: 4 females and 4 males) Yes Beylaqan Ashaghi Chemenli 1 meeting (11 participants: 5 females and 6 males) No Ganja￾Qazakh Samukh Ali Bayramli 1 meeting (8 participants: 4 females and 4 males) Yes Goranboy Seyidler 1 meeting (9 participants: 4 females and 5 males) Yes Qushchular 1 meeting (6 participants: 3 females and 3 males) Yes Xoylu No meeting Yes Quba￾Khachmaz Quba Pustaqasim 1 meeting (4 participants: 4 males ) Yes Alpan 1 meeting (7 participants: 1 female and 6 male ) Yes Rustov 1 meeting (5 participants: 5 males) No Shabran Amirkhanli 1 meeting (7 participants: 2 female and 5 male) Yes Surra 1 individual meeting (1 male) Yes Khachmaz Aghayazi Buduq 1 meeting (7 participants: 2 females and 5 males) Yes Total 8 Rayons 15 Communities 98 participants: 31 females and 67 males Resource Centers and NGOs Region Town CSOs Group Meeting Ganja￾Qazakh Ganja Ganja Euro-Atlantic Information Centre (CSO) 1 group meeting (7 participants: 3 male and 4 females) Ganja Regional Advice Centre PU (CSO) Women in Civil Society (CSO) “Contact” Regional Development PU (CSO) Bridge to the Future YPU (CSO) The Society of Disable Women (CSO) Youth Unit of the Society of Disable Women (CSO) Quba￾Khachmaz Quba NGO Forum (CSO) 1 group meeting (9 participants: 4 males and 5 females) Quba Regional Resource Centre (RRC) Young Talents Public Union (CSO) Regional Media Development PU (CSO) Karabakh Veterans and Martyrs PU (CSO) Reliable Future PU (CSO) Total: 13 CSOs, 16 participants, 7 males and 9 females SEDA Regional Offices in Imishli, Ganja, and Quba Region Town Group Meeting Aran Imishli 1 group meeting (5 participants: 3 males and 2 females). Ganja-Qazakh Ganja 1 group meeting (6 participants: 2 males and 4 females) Quba-Khachmaz Quba 1 group meeting with staff (4 participants: 2 males and 2 females) Total 15 participants: 7 males and 8 females. Key Informant Interviews Data Source Key Informant Interview Group Discussion USAID 1 1 group meeting (5 participants) EWMI/SEDA 3 1 group meeting (5 participants) UMID 2 ICNL/MG Consulting 1 SPPU 1 Forum of NGOs/ Baku Resource Center 1 NGO Council 1 Ministry of Economy and Industry 4 Ministry of Education in Quba 1 Beylaqan Youth Enlightenment and Social Development Public Union (grant recipient) 1 World Bank AZRIP Project 1 Constitutional Research Fund 1 Total 18 2 Annex 7. CDC Profile CDC Profiles Presented below is a summary profile of the SEDA Program Community Development Councils (CDCs) based on data provided by EWMI/SEDA on 103 councils. The SEDA approach to working in local communities is founded on the establishment of CDCs as a means of building the capacities of local populations to prioritize community needs and devise development solutions through the application of participatory methods. Following a common protocol, CDCs were established in each participating community based on an anonymous voting process by local citizens. The majority of CDCs (76%) have nine or ten members, with 19.2% having 11 or more members and a small number (4.9%) having eight or fewer members. As presented in the table below, the number of established CDCs reflects the intensity of work in the SEDA target regions. As the table shows, the Aran region hosts the greatest number CDCs (42 or 40.8% of the national total), followed by Quba (36 or 35%), and Ganja (25 or 24.3% of the total CDCs). Analysis of communities show that in 86.4% of these communities no local/community NGO is active. Region Answer Options Response Percent Response Count Quba 35,0% 36 Ganja 24,3% 25 Aran 40,8% 42 answered question 103 While almost one-third (32.7%) of SEDA local communities were “pre-mobilized – that is, had experienced community development activities prior to the SEDA intervention – the majority of participating communities had not. As depicted in the pie-chart below, more than half of the CDCs (55%) were established in 2014 and 2015, reflecting the increase in SEDA activities in the past two years, while almost all of the rest were established in 2012 and 2013. There are five CDC that were established even earlier (between 1995 and 2010). Only three CDCs have been dissolved and no longer exist. 26% 18% 20% 35% Chart 1. When was the council created? 2012 2013 2014 2015 Almost all CDCs meet regularly, except those which have not continued functioning. Out of functional CDCs, 18.3% have succeeded in getting funding from other sources, besides SEDA. In the vast majority communities where SEDA is active, there are no active NGOs (86.4%) of any kind. The CDCs generally have a good representation of women. The majority (55.8%) of the CDCs are comprised of 41-50% women members; about one-third (31.7%) have 20-40% female membership; and 12.5% are comprised of over 50% of women members. Each SEDA CDC elects a head or CDC president. As depicted in Chart 2 below, one-fourth of CDC presidents are local administration employees, while other one fourth are local teachers. The vast majority (84.6%) of CDC heads are male, with over half the leaders being over 50 years of age (64.1%), 30.1% are between ages 30 and 50, and 5.8% under 30 years old. 01% 00% 27% 24% 06% 05% 00% 01% 05% 00% 00% 01% 16% 00% 14% Chart 2. President of the Council Head of the local Executive Committee Religious leader Local administration employee Local teacher Director of education institution Local businessman/woman Local doctor Local medical staff Farmer Student NGO representative Unskilled Worker Skilled worker (secondary or higher degree working in some enterprise or else) Unemployed Retired While the greatest number of CDC members are over the age of 50, as Chart 3 depicts, members’ ages vary significantly. As presented in Chart 4 below, CDC members vary in occupation, including students, retired persons, farmers, teachers, businessmen/women and unemployed local residents. Chart 3. Age of Council Members under 30 31-50 over 50 Cooperation with SEDA The majority of CDCs (73.8%), developed more than two infrastructure project proposals which were submitted to SEDA. The majority of CDCs (73.1%) submitting proposals, received a grant for a single infrastructure project; 15.4% received two grants and 4.8% three or more; 6.7% have not yet received approval for a proposed project. Over 50% of projects are not yet finished, while 44% are finished by the time of the Evaluation. Projects address a variety of communal issues. About one-third (34%) involve road construction or health (30.8%), followed by schools or pre-school (32%), water (25%), or social services. As presented in Chart 5 below, there are also projects in the areas of recreation, bridges, agriculture, sports, electricity, and business development. 4% 1% 15% 17% 7% 2% 3% 4% 11% 0% 0% 8% 11% 9% 8% Chart 4. Profession of CDC Members Head of the Local Executive Committee Religious Leader Local administration employee Local teacher Director of education institutions Local businessman/woman Local doctor Local medical staff Farmers Student NGO representative Unskilled worker Skilled workers (secondary or higher degree working in some enterprise or else) Unemployed Retired Although the majority of CDCs (81.7%) did not receive funding for projects subsequent to completing a SEDA infrastructure project,18.3% succeeded securing project funding from other sources. 19% 22.40% 2.58% 1.72% 18% 2.58% 8.60% 11.20% 7.70% 5% 0.80% 0% 5% 10% 15% 20% 25% Chart 5. What did the approved projects cover? Annex 8. Survey of CDC and CSO Members Survey of CDC Members – collected during the field work During the field work, the Evaluation Team collected survey questionnaires from a total of 86 CDC members that the team met during the field visits to target communities included in the sample. The CDCs see that SEDA’s performance towards improving the ways local community members, community organizations and local government officials work together to create socio-economic development and address key needs and priorities in their villages and regions is extremely good or good (93%). There was three CDC members who assessed the performance as poor or not so well. The CDC members rate trainings they received through SEDA Project as extremely effective or effective (93%), while three members rate them as not so effective. Asked about the transparency of process for development of the project to the involvement and input of all interested community member, responses vary, as presented in Graph 6 below. While 85% mark it open, 2.2% mark it as closed and 4.5% average. The rest, 7.9% rate it as not so closed. CDCs confirm that they are likely to develop and implement other such projects in the future, with only 4.6% of members rating it unlikely or extremely unlikely. Majority of members also confirm that they will remain being active in CDC (87.7%), while the remaining 12.3% respond they will maybe continue. 02% 05% 08% 85% Graph 6. On a five point scale in which one represents extremely closed and five, extremely open, how would you rate the openness of the process for development of infrastructure project to the involvement and input of all interested community members Extremely closed Closed Average Not so closed Open Survey for CSOs in regions – collected by Team Total of 16 questionnaires have been collected during the field visits to the communities from CSOs active in the regions. All surveyed CSOs see that SEDA’s performance towards improving the ways local community members, community organizations and local government officials work together to create socio-economic development and address key needs and priorities in their villages and regions is extremely good or good. Trainings and capacity building activities that CSOs received are mainly rated as extremely effective or effective, while 6.3% of respondents rate it as average (See Graph 1 below). Closely related to that, CSOs see overall effectiveness of SEDA in building capacities of CSOs are very effective or effective (87.5%), while 12.5% rate it as average. On the other hand, there is a bit of variation in ratings of the effectiveness of Resource and Training Centers for strengthening CSOs. 87% consider it very effective or effective, 6.3% respectively rate it as average or not so effective (Graph 2). Graph 1. On a five-point scale in which one represents Not effective at all and five, Extremely effective, please rate the effectiveness of the training courses in building the capacity of your organisation: Extremely effective Effective Average Not so effective Not at all effective Graph 2. How effective are the Regional NGO Resource and Training Centers in strengthening NGOs and other local civil society organizations? Very effective Effective Average Not so effective Not at all effective Annex 9. Survey with CSOs Participating in SPPU Training Survey with CSOs participating in SPPU trainings The survey was distributed among 20 NGOs that had participated in the trainings organized by SPPU. From the profile of respondents shown in the Chart 1 below, it may be seen that over 60% of respondents were directors of NGOs, while the rest are staff members or volunteers. Survey shows that organizations participating in trainings are active in various thematic areas as shown in Chart 2 below. 65% 10% 00% 20% 05% Chart 1. Respondent's role in the organization Director Administrative/finance staff member Supervisor Staff Member Volunteer 00% 10% 20% 30% 40% 50% 60% Advocacy and monitoring of policies Protection of women’s rights Socio-economic development Education Health Youth Services Community engagement Sports and recreation Arts Chart 2. Main areas of work or focus of the organization? NGOs see many external factors that affect their work (See Chart 3 below). Main factors influencing the work are political and socio-economic factors, but also regional isolation, for organizations working in smaller communities. Another important factor mentioned by respondents is the tradition of non-participation in local governance (12% or respondents). Most NGOs participating in trainings have not received grants from SEDA, except in case of five organisations participating in the survey. Those who did receive the grant, rated the granting and implementation experience as excellent or good. NGOs participated in a range of capacity building events offered by the Project. Most of them participated in the trainings on project planning and implementation trainings (74%), while over a half of respondents participated in advocacy or organizational capacity building trainings. Overview of trainings is presented in Chart 4 below. All training participants rate the trainings as very useful (73.7%) or useful (26.3%). Most beneficial for trainees was learning about decision-making, analysis, civil society, gaining experience and knowledge. Also, respondents mention networking and assistance in project preparation and organization. Over a half of respondents participated in study tours (54%) which they rated either as very useful (83%), or useful or average (8,3% for each category). Almost 80% of participants took part in learning circles as well and rated it as very useful or useful experience (93,3%) or average (6,7%). 47% 47% 35% 06% 06% 12% 00% 05% 10% 15% 20% 25% 30% 35% 40% 45% 50% Political situation in the country Socio-economic factors and poverty Regional isolation Competitiveness with other CSOs Governmental opposition or indifference Tradition of non￾participation in local governance Chart 3. What external factors influence the work of your CSO? Training participants see that the capacity building support offered by the Project has brought positive changes in the way their organization operates in majority of areas, as presented in Chart 5 below. Some organizations also report that they have increased their funding sources to some extent to some extent. Most interestingly, organizations state that they are now more focused on what they want to achieve, which may point to Project contribution to empowerment of NGOs and assisting them to reflect and find their niche. Also, organizations benefited from networking with other organizations through the training activities, which may be a positive investment in future joint actions or initiatives. 53% 58% 42% 26% 47% 11% 74% 37% 37% 11% 00% 05% 11% 00% 10% 20% 30% 40% 50% 60% 70% 80% SEDA’s CSO Advocacy … Organizational Capacity… Conducting meaningful… Decision making Needs assessment Stakeholder analysis Project planning, design and… Environmental impact Cost-benefit analysis Vocational training Support projects in animal… Modern crop production… None Chart 4. In which SEDA-sponsored trainings did you participate? Overall, majority of organizations rate their cooperation with SEDA as excellent (89%), while 5,6% respectively rate it either as good or average. Almost 80% of NGOs see that SEDA has contributed quite a bit to increasing local civic participation and collaboration between civil society and local government, while 16% of NGOs rate is as tremendous contribution. The remaining 5,3% of respondents see that SEDA has somewhat contributed to the increase. 0 2 4 6 8 10 12 14 16 We are more focused on what we want to achieve We have a better strategy We have more professional expertise in our organization thanks to trainings we attended Advocacy initiatives are now better designed We understand power dynamics and power analysis better (for advocacy) We have skills to conduct better context analysis and research on a topic of interest We have more funding sources We have better relationship with the authorities We have better cooperation with other organizations that participated in the trainings We are better known and respected in our community We increased coverage of gender issues We have improved our monitoring and evaluation of advocacy capacity Chart 5. Changes in the organization as a result of the capacity building support received from SEDA Project No, it is worse No change Yes, a little Yes, definitely One respondent also mentioned the SEDA’s work in village of Henna, where the water line was repaired, and now functions smoothly, while also other projects carried out in the region are proving to have the best effect. The respondent concludes: “SEDA - appreciate the excellent performance”. 16% 79% 05% 00% 00% Chart 6. In your opinion, to what extent has the SEDA Project contributed to increasing local civic participation and collaboration between civil society and local government? A tremendous amount Quite a bit Somewhat Slightly Not at all Annex 10. Data from CDCs Community Development Councils Question 1 Region Answer Options Response Percent Response Count Guba 35.0% 36 Ganja 24.3% 25 Aran 40.8% 42 answered question 103 skipped question 1 Community Development Councils Question 2 Number of members in the Council Answer Options Response Percent Response Count Up to 5 1.0% 1 6-7 1.0% 1 7-8 2.9% 3 9-10 76.0% 79 more than 10 19.2% 20 answered question 104 skipped question 0 Community Development Councils Question 3 Is this a pre-mobilised community? Answer Options Response Percent Response Count Yes 32.7% 34 No 67.3% 70 answered question 104 skipped question 0 Community Development Councils Question 4 When was the council created? Answer Options Response Percent Response Count 2012 26.3% 26 2013 18.2% 18 2014 20.2% 20 2015 35.4% 35 Other (please specify) 5 answered question 99 skipped question 5 Number Response Date Other (please specify) Categories 1 Dec 8, 2015 3:13 PM 2010 2 Dec 8, 2015 12:57 PM 2006 3 Dec 8, 2015 8:55 AM 1995 4 Dec 8, 2015 8:05 AM 2010 5 Dec 8, 2015 6:47 AM 2010 Community Development Councils Question 5 What is the % of women in the CDC? Answer Options Response Percent Response Count Less than 5% 0.0% 0 5-10% 0.0% 0 11-20% 1.9% 2 21-30% 6.7% 7 31-40% 24.0% 25 41-50% 54.8% 57 51-60% 12.5% 13 61-70% 0.0% 0 71-80% 0.0% 0 more than 80% 0.0% 0 answered question 104 skipped question 0 Community Development Councils Question 6 President of the Council Answer Options Response Percent Response Count Head of the local Executive Committee 1.0% 1 Religious leader 0.0% 0 Local administration employee 26.9% 28 Local teacher 24.0% 25 Director of education institution 5.8% 6 Local businessman/woman 4.8% 5 Local doctor 0.0% 0 Local medical staff 1.0% 1 Farmer 4.8% 5 Student 0.0% 0 NGO representative 0.0% 0 Unskilled Worker 1.0% 1 Skilled worker (secondary or higher degree working in some enterprise or else) 16.3% 17 Unemployed 0.0% 0 Retired 14.4% 15 Other (please specify) 0 answered question 104 skipped question 0 Community Development Councils Question 7 President's gender Answer Options Response Percent Response Count Male 84.6% 88 Female 15.4% 16 answered question 104 skipped question 0 Community Development Councils Question 8 President age Answer Options Response Percent Response Count up to 30 5.8% 6 31-50 30.1% 31 over 50 64.1% 66 answered question 103 skipped question 1 Member 1 Member 2 Member 3 Member 4 Member 5 Member 6 Member 7 Member 8 Member 9 Member 10 Response Count 0 1 1 13 48 27 9 2 2 0 103 0 6 19 53 16 6 0 1 1 0 102 103 1 Community Development Councils skipped question Gender of the Council Members Answer Options Male Female answered question Community Development Councils Question 10 Age of Council members Answer Options Response Count under 30 47 31-50 100 over 50 102 answered question 104 skipped question 0 Age of Council members Response Count under 30 under 30 31-50 31-50 over 50 over 50 Age of Council Members under 30 31-50 over 50 Community Development Councils Question 11 Member of the Council Head of the local Executive Committee 23 Religious leader 4 Local administration employee 76 Local teacher 90 Director of education institution 11 Local businessman/woman 34 Local doctor 16 Local medical staff 20 Farmer 55 Student 2 NGO representative 1 Unskilled Worker 44 Skilled worker (secondary or higher degree working in some enterprise or else) 57 Unemployed 46 Retired 43 Other (please specify) 0 answered question 104 skipped question 0 Chart Title NGO representative Unskilled worker Skilled workers (secondary or higher degree working in some enterprise or else) Unemployed Community Development Councils Question 12 How many projects did the CDC develop for SEDA? Answer Options Response Percent Response Count 1 21.4% 22 2 4.9% 5 more than 2 73.8% 76 answered question 103 skipped question 1 How many projects did the CDC develop for SEDA? 1 2 more than 2 Community Development Councils Question 13 How many of these nominated projects were approved? Answer Options Response Percent Response Count 1 73.1% 76 2 15.4% 16 more than 2 4.8% 5 None 6.7% 7 answered question 104 skipped question 0 How many of these nominated projects were approved? 1 2 more than 2 None Community Development Councils Question 14 How many of these approved projects are implemented? Answer Options Response Percent Response Count 1 49.0% 51 2 8.7% 9 more than 2 1.0% 1 None, implementation has not started yet 41.3% 43 answered question 104 skipped question 0 How many of these approved projects are implemented? 1 2 more than 2 None, implementation has not started yet Community Development Councils Question 15 How many of the approved projects are finished? Answer Options Response Percent Response Count none 52.5% 52 1 44.4% 44 2 3.0% 3 more than 2 0.0% 0 answered question 99 skipped question 5 How many of the approved projects are finished? none 1 2 more than 2 Community Development Councils Question 16 What areas do the nominated projects cover? Answer Options Response Count Water utility 23 Roads 31 Bridges 3 Recreation 6 Health 28 Pre-school 11 Schools 20 Social service 21 Other 40 answered question 91 skipped question 13 0 5 10 15 20 25 30 35 40 45 Water utility Roads Bridges Recreation Health Pre-school Schools Social service Other What areas do the nominated projects cover? Response Count Community Development Councils Question 17 What did the approved projects cover? Answer Options Response Percent Response Count Water utility 25.6% 22 Roads 30.2% 26 Bridges 3.5% 3 Other transport 0.0% 0 Recreation 0.0% 0 Health 24.4% 21 Pre-school 3.5% 3 Schools 11.6% 10 Social service 15.1% 13 Other (please specify) 19 answered question 86 skipped question 18 Number Response Date Other (please specify) Categories 1 Dec 9, 2015 1:13 PM Electricity 2 Dec 9, 2015 1:03 PM Agriculture 3 Dec 9, 2015 12:03 PM İnfrastructure 4 Dec 9, 2015 7:06 AM Agriculture 5 Dec 9, 2015 6:54 AM Electricity 6 Dec 9, 2015 6:42 AM Electricity 7 Dec 8, 2015 6:30 PM Business 8 Dec 8, 2015 6:13 PM Sport 9 Dec 8, 2015 5:52 PM electricity 10 Dec 8, 2015 5:48 PM Sport 11 Dec 8, 2015 5:19 PM electricity 12 Dec 8, 2015 3:52 PM Electricity 13 Dec 8, 2015 1:43 PM Agriculture 14 Dec 8, 2015 12:57 PM Agriculture 15 Dec 8, 2015 11:43 AM Agriculture 16 Dec 8, 2015 8:05 AM Electricity 17 Dec 8, 2015 7:48 AM Electricity 18 Dec 8, 2015 7:06 AM Electricity 19 Dec 8, 2015 6:10 AM Agriculture 0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% What did the approved projects cover? 19% 22.40% 2.58% 1.72% 18% 2.58% 8.60% 11.20% 7.70% 5% 0.80% 0% 5% 10% 15% 20% 25% Chart Title Community Development Councils Question 18 Did such form of local initiative (community action group) exist before? Answer Options Response Percent Response Count Yes 28.2% 29 No 71.8% 74 answered question 103 skipped question 1 Did such form of local initiative (community action group) exist before? Yes No Community Development Councils Question 19 Does the CDC still exist? Answer Options Response Percent Response Count Yes 97.1% 101 No 2.9% 3 answered question 104 skipped question 0 Does the CDC still exist? Yes No Community Development Councils Question 20 Has the council (CDC) met within the last three months? Answer Options Response Percent Response Count Yes 97.1% 101 No 2.9% 3 answered question 104 skipped question 0 Has the council (CDC) met within the last three months? Yes No Community Development Councils Question 21 Did the Council get any funding for any projects they developed after the SEDA project? Answer Options Response Percent Response Count Yes 18.3% 19 No 81.7% 85 answered question 104 skipped question 0 Did the Council get any funding for any projects they developed after the SEDA project? Yes No Community Development Councils Question 22 Are there other local/community NGOs working in this community? Answer Options Response Percent Response Count Yes 13.6% 14 No 86.4% 89 answered question 103 skipped question 1 Are there other local/community NGOs working in this community? Yes No Annex 11. SPPU Survey Data İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 1 Your role in the organization Answer Options Response Percent Response Count Director 65.0% 13 Administrative/finance staff member 10.0% 2 Supervisor 0.0% 0 Staff Member 20.0% 6 Volunteer 5.0% 1 Other (organisation member) 2 answered question 20 skipped question 0 Number Response Date Digər, ətraflı qeyd edin Categories 1 Dec 10, 2015 9:41 AM dinleyici kimi istirak etmisem 2 Dec 9, 2015 5:38 AM Abayev Qəşəm Məmməd oğlu Xınalıq Turizmin İnkişafına Dəstək İctimai birliyin sədri Sizin təşkilatdakı rolunuz/Your role in the organization Director Administrative/finance staff member Supervisor Staff Member Volunteer İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 2 What are the main areas of your work or focus of your organization? Answer Options Response Percent Response Count Advocacy and monitoring of policies 26.3% 5 Protection of women’s rights 21.1% 4 Socio-economic development 52.6% 10 Education 31.6% 6 Health 10.5% 2 Youth Services 42.1% 8 Community engagement 31.6% 6 Sports and recreation 21.1% 4 Arts 5.3% 1 Digər, zəhmət olmasa izah edin 4 answered question 19 skipped question 1 Number Response Date Digər, zəhmət olmasa izah edin Categories 1 Dec 10, 2015 10:23 AM Region QHT-lərinin maariflndirilməsi, təlim-seminarların təşkili, əməkdaşlıq 2 Dec 10, 2015 9:41 AM genlerin inkisafina destek 3 Dec 9, 2015 3:53 PM Yerli QHT-lərə və icmalara yardım 4 Dec 9, 2015 5:38 AM Regionlarda əhalinin sosial -iqtisadi inkişafını artırmaq məqsədilə birgə işlər təşkil etmək 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% Advocacy and monitoring of policies Protection of women’s rights Socio-economic development Education Health Youth Services Community engagement Sports and recreation Arts Sizin əsas fəaliyyət sahələriniz hansılardır və ya təşkilatınızın əsas iş istiqaməti?/What are the main areas of your work or focus of your organization? İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 3 What external factors influence the work of your CSO? Answer Options Response Percent Response Count Political situation in the country 47.1% 8 Socio-economic factors and poverty 47.1% 8 Regional isolation 35.3% 6 Competitiveness with other CSOs 5.9% 1 Governmental opposition or indifference 5.9% 1 Tradition of non-participation in local governance 11.8% 2 Digər 1 answered question 17 skipped question 3 0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0% 50.0% Political situation in the country Socio-economic factors and poverty Regional isolation Competitiveness with other CSOs Governmental opposition or indifference Tradition of non￾participation in local governance VCT-ın fəaliyyətinə təsir edən xarici faktorlar nələrdir? Zəhmət olmasa sizə uyğun olanları seçin What external factors influence the work of your CSO? İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 4 Bundan əvvəl və ya hal-hazırda sizin təşkilatınız SEDA-dan qrant və ya hər hansı bir dəstək almışdırmı? Does/did your organization receive a grant or other support from SEDA? Answer Options Response Percent Response Count Bəli/ Yes 25.0% 5 Xeyr/ No 75.0% 15 answered question 20 skipped question 0 Bundan əvvəl və ya hal-hazırda sizin təşkilatınız SEDA-dan qrant və ya hər hansı bir dəstək almışdırmı? Does/did your organization receive a grant or other support from SEDA? Bəli/ Yes Xeyr/ No İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 5 Əgər almışdırsa, bu dəstək nə kimi dəstək olub? If yes, which kind Answer Options Response Percent Response Count İcmanın Özünəyardım Qrant Proqramı/ Community Self￾Help Grants program 33.3% 1 SEDA İctimai Fəallıq Qrant (layihəsi)/ SEDA Civic Engagement Grants 0.0% 0 İcmalar üçün Sosial-İqtisadi Layihələr/ Community Socio￾Economic Projects 66.7% 2 Əmin deyiləm (Fikrim yoxdur)/ not sure 0.0% 0 Digər, zəhmət olmasa izah edin 4 answered question 3 skipped question 17 Number Response Date Digər, zəhmət olmasa izah edin Categories 1 Dec 10, 2015 4:23 PM mərkəzin persoanalina maddi dəstək 2 Dec 10, 2015 10:23 AM layihə uduldu amma müxtəlif səbəblərdən maliyyələşdirilmədi. 3 Dec 9, 2015 3:53 PM Resurs mərkəzlərinin dəstəklənməsi 4 Dec 8, 2015 12:10 PM Heç bir qrant almamışıq Əgər almışdırsa, bu dəstək nə kimi dəstək olub? If yes, which kind İcmanın Özünəyardım Qrant Proqramı/ Community Self￾Help Grants program SEDA İctimai Fəallıq Qrant (layihəsi)/ SEDA Civic Engagement Grants İcmalar üçün Sosial-İqtisadi Layihələr/ Community Socio￾Economic Projects Əmin deyiləm (Fikrim yoxdur)/ not sure İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 6 SEDA çərçivəsində əldə etdiyiniz grantlarla bağlı qazandığınız təcrübənizi necə qiymətləndirərdiniz?How would you rate your experience with the grant(s) you have received from SEDA? Answer Options Response Percent Response Count Əla/ Excellent 50.0% 2 Yaxşı/ Good 50.0% 2 Orta/ Average 0.0% 0 Zəif/ Poor 0.0% 0 Çox pis/ Terrible 0.0% 0 answered question 4 skipped question 16 SEDA çərçivəsində əldə etdiyiniz grantlarla bağlı qazandığınız təcrübənizi necə qiymətləndirərdiniz?How would you rate your experience with the grant(s) you have received from SEDA? Əla/ Excellent Yaxşı/ Good Orta/ Average Zəif/ Poor Çox pis/ Terrible İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 7 What were the three main achievement of your project? Please list below: Answer Options Response Count 3 answered question 3 skipped question 17 Number Response Date Response Text Categories 1 Dec 10, 2015 4:56 PM Yerli QHT-lərə dəstək Şəbəkələşmə İnkişaf 2 Dec 10, 2015 6:17 AM əhali arasinda aparilan maarifləndirmə işləri. 3 Dec 9, 2015 3:53 PM 1. Regional QHT-lərin və icmaların inkişafı 2. Dövlət qurumları ilə əməkdaşlıq 3. Regional şəbəkənin yaradılması İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 8 In which SEDA-sponsored trainings did you participate? Answer Options Response Percent Response Count SEDA’s CSO Advocacy Training Program 52.6% 10 Organizational Capacity Development Workshops 57.9% 11 Conducting meaningful meetings 42.1% 8 Decision making 26.3% 5 Needs assessment 47.4% 9 Stakeholder analysis 10.5% 2 Project planning, design and implementation 73.7% 14 Environmental impact 36.8% 7 Cost-benefit analysis 36.8% 7 Vocational training 10.5% 2 Support projects in animal husbandry and veterinary care 0.0% 0 Modern crop production practices 5.3% 1 None 10.5% 2 Digər, zəhmət olmasa sadalayın 0 answered question 19 skipped question 1 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% SEDA’s CSO Advocacy … Organizational Capacity… Conducting meaningful… Decision making Needs assessment Stakeholder analysis Project planning,… Environmental impact Cost-benefit analysis Vocational training Support projects in… Modern crop production… None SEDA tərəfindən maliyyələşdirilən hansı təlimlərdə iştirak etmisiniz? In which SEDA-sponsored trainings did you participate? İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 9 How would you assess the trainings you received from SEDA? Answer Options Response Percent Response Count Very useful 73.7% 14 Useful 26.3% 5 Average 0.0% 0 Not very useful 0.0% 0 Not at all useful 0.0% 0 answered question 19 skipped question 1 SEDA-nın keçirdiyi təlimləri necə qiymətləndirərdiniz? How would you assess the trainings you received from SEDA? Very useful / Çox faydalı Useful / Faydalı Average / Orta Not very useful / Çox az faydalı Not at all useful / Tamamilə faydasız Very useful Useful Average Not very useful Not at all useful İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 10 Sizin təlimlərdə öyrəndiyiniz ən faydalı şey nədir? What was the most beneficial thing you learned in the trainings? Answer Options Response Count 13 answered question 13 skipped question 7 Number Response Date Response Text Categories 1 Dec 14, 2015 9:07 AM Qərar qəbul etmə,Analiz, İctimaiyyətlə münasivət,vətəndaş cəmiyyətinin inkişafı və s. 2 Dec 12, 2015 9:36 AM tecrube biliklerin artirilmasi 3 Dec 11, 2015 4:54 PM Təcrübə qazanmaq, biliklərin artırılması 4 Dec 10, 2015 5:51 PM 1.Təcrübə 2. Şəbəkə 5 Dec 10, 2015 4:56 PM Təcrübə Biliklərin genişləndirilməsi 6 Dec 10, 2015 4:23 PM biliklərin genişləndirilməsi 7 Dec 10, 2015 10:23 AM Organizational Capacity Development Workshops Conducting meaningful meetings Decision making Needs assessment Stakeholder analysis Project planning, design and implementation Environmental impact 8 Dec 10, 2015 9:41 AM Ehtiyaclarin odenilmesi 9 Dec 10, 2015 6:17 AM layihənin hazirlanması ilə bağlı yardımın alınması. 10 Dec 9, 2015 3:53 PM Biliklərin təkmilləşdirilməsi Təcrübənin artması Təlimçi kimi fəaliyyətin gücləndirilməsi 11 Dec 8, 2015 12:00 PM Layihe hazirlanmasi, qiymetlendirilmesi ve tərəfdaş təşkilatlarla əlaqə 12 Dec 8, 2015 11:43 AM Hər şey. 13 Dec 8, 2015 11:19 AM ehtiyacların qiymətləndirilməsi və təhlil İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 11 Siz tanışlıq (işgüzar) səfərlərində iştirak etmisinizmi? Did you participate in study tours? Answer Options Response Percent Response Count Bəli / Yes 55.0% 11 Xeyr / No 45.0% 9 answered question 20 skipped question 0 Siz tanışlıq (işgüzar) səfərlərində iştirak etmisinizmi? Did you participate in study tours? Bəli / Yes Xeyr / No İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 12 İştirak etmisinizsə siz bunu necə qiymətləndirərdiniz? If yes, how would you rate it? Answer Options Response Percent Response Count Çox faydalı / very useful 83.3% 10 Faydalı / Useful 8.3% 1 Orta / Average 8.3% 1 Çox az faydalı / Not very useful 0.0% 0 Tamamilə faydasız / Not at all useful 0.0% 0 answered question 12 skipped question 8 İştirak etmisinizsə siz bunu necə qiymətləndirərdiniz? If yes, how would you rate it? Çox faydalı / very useful Faydalı / Useful Orta / Average Çox az faydalı / Not very useful Tamamilə faydasız / Not at all useful İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 13 VCT-lər üçün keçirilən silsilə təlimlərdə (tədris dairəsi) iştirak etmisinizmi? Did you take part in learning circle for CSOs? Answer Options Response Percent Response Count Bəli / yes 78.9% 15 Xeyr / no 21.1% 4 answered question 19 skipped question 1 VCT-lər üçün keçirilən silsilə təlimlərdə (tədris dairəsi) iştirak etmisinizmi? Did you take part in learning circle for CSOs? Bəli / yes Xeyr / no İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 14 İştirak etmisinizsə necə qiymətləndirərdiniz? If yes, how would you rate it? Answer Options Response Percent Response Count Çox faydalı / Very useful 73.3% 11 Faydalı / Useful 20.0% 3 Orta / Average 6.7% 1 Çox az faydalı / Not very useful 0.0% 0 Tamamilə faydasız / Not at all useful 0.0% 0 answered question 15 skipped question 5 İştirak etmisinizsə necə qiymətləndirərdiniz? If yes, how would you rate it? Çox faydalı / Very useful Faydalı / Useful Orta / Average Çox az faydalı / Not very useful Tamamilə faydasız / Not at all useful İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 15 Please assess the changes in your organization as a result of the capacity building support you have received from SEDA Project: Answer Options Yes, definitely Yes, a little No change No, it is worse Response Count We are more focused on what we want to achieve 14 0 1 0 15 We have a better strategy 11 2 1 1 15 We have more professional expertise in our organization thanks to trainings we attended 10 4 1 0 15 Advocacy initiatives are now better designed 13 2 0 0 15 We understand power dynamics and power analysis better (for advocacy) 10 4 0 0 14 We have skills to conduct better context analysis and research on a topic of interest 13 1 0 0 14 We have more funding sources 5 7 1 0 13 We have better relationship with the authorities 13 1 0 0 14 We have better cooperation with other organizations that participated in the trainings 14 0 0 0 14 We are better known and respected in our community 13 1 0 0 14 We increased coverage of gender issues 12 2 0 0 14 We have improved our monitoring and evaluation of advocacy capacity 13 1 0 0 14 answered question 16 skipped question 4 0 5 10 15 Please assess the changes in your organization as a result of the capacity building support you have received from SEDA Project: No, it is worse No change Yes, a little Yes, definitely 0 2 4 6 8 10 12 14 16 We are more focused on what we want to… We have more professional… We understand power dynamics and power… We have more funding sources We have better cooperation with… We increased coverage of gender… SEDA layihəsi çərçivəsində Bəli, sözsüz (tamamilə) / Yes, definitely Bəli, çox az / Yes, a little Dəyişiklik yoxdur / No change Xeyr, bu daha da pisdir / No, it is worse İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 16 Ümumi olaraq SEDA ilə əməkdaşlığınızı necə qiymətləndirərdiniz? Overall, how would you assess your cooperation with SEDA? Answer Options Response Percent Response Count Əla / Excellent 88.9% 16 Yaxşı / Good 5.6% 1 Orta / Average 5.6% 1 Zəif / Poor 0.0% 0 Çox pis / Terrible 0.0% 0 answered question 18 skipped question 2 Ümumi olaraq SEDA ilə əməkdaşlığınızı necə qiymətləndirərdiniz? Overall, how would you assess your cooperation with SEDA? Əla / Excellent Yaxşı / Good Orta / Average İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 17 In your opinion, to what extent has the SEDA Project contributed to increasing local civic participation and collaboration between civil society and local government? Answer Options Response Percent Response Count A tremendous amount 15.8% 3 Quite a bit 78.9% 15 Somewhat 5.3% 1 Slightly 0.0% 0 Not at all 0.0% 0 answered question 19 skipped question 1 Sizin fikrinizcə SEDA layihəsi yerli vətəndaş fəallığının və vətəndaş cəmiyyəti ilə yerli icra ha A tremendous amount Quite a bit Somewhat Slightly Not at all İcmalara Xidmət Təşkilatının (CSO) Sorğusu Question 18 Digər qeydlər / Other comments Answer Options Response Count 10 answered question 10 skipped question 10 Numbe r Response Date Response Text Categories 1 Dec 10, 2015 5:51 PM təşəkkür 2 Dec 10, 2015 4:56 PM uğurlar 3 Dec 10, 2015 4:23 PM Dağa sıx əməkdaşlığa ehtiyac var 4 Dec 10, 2015 7:19 AM Salam! Xahis ederdik "Bereket" İB-de emekdasliqa devet edesiniz. 5 Dec 10, 2015 6:17 AM ümid edirki SEDA ilə əməkdaşıqmiz davam edər. 6 Dec 9, 2015 3:53 PM Əməkdaşlığa görə təşəkkür edirik 7 Dec 9, 2015 8:56 AM SEDA ilə əməkdaşlığımızın daha uzun ömürlü olmasını diləyirəm. 8 Dec 9, 2015 5:38 AM Xınalıq Turizmin İnkişafına Dəstək İctimai Birliyi 2013-ci ildə layihələrin icrası ilə əlaqadar SEDA - ya iki layihə vermişdir, layihələrin heş birini maliyyələşdirmədi. Amma 2011 ci ildə Xınalıq kəndində SEDA - nın layihəsi hesabına Xınalıq kəndinin yeni su xətti təmir və bərpa olundu və günə kimi problemsiz işləyir. Rayonda həyata keçirilmiş digər layihələrində yaxşı effekt verdiyinin şahidiyik, buna görə SEDA - ni fəaliyyətinin ƏLA qiymətləndirirəm 9 Dec 8, 2015 12:00 PM Beynalxalq görüşlərin keçirilməsi 10 Dec 8, 2015 11:19 AM Uğurlar və VC təşkilatlarının iştirakı ilə davamlılıq Annex 12. List of Documents Reviewed Bibliography of Documents and Sources Reviewed USAID Program Description: Socio Economic Development Activity List of Trainings Provided by SEDA USAID Civil Society Organizational and Advocacy Capacity Assessment East West Management Institute (2013) Post Training Impact Assessment Aran and Guba-Khachmaz East West Management Institute (2012) First Annual Report East West Management Institute (2013) Second Annual Report East West Management Institute (2014) Third Annual Report East West Management Institute (2015) Fourth Annual Report East West Management Institute (2011) First Quarterly Report East West Management Institute (2012) Second Quarterly Report East West Management Institute (2012) Third Quarterly Report East West Management Institute (2012) Fourth Quarterly Report East West Management Institute (2012) Fifth Quarterly Report East West Management Institute (2013) Sixth Quarterly Report East West Management Institute (2013) Seventh Quarterly Report East West Management Institute (2013) Eighth Quarterly Report East West Management Institute (2013) Ninth Quarterly Report East West Management Institute (2014) Tenth Quarterly Report East West Management Institute (2014) Eleventh Quarterly Report East West Management Institute (2014) Twelfth Quarterly Report East West Management Institute (2014) Thirteenth Quarterly Report East West Management Institute (2015) Fourteenth Quarterly Report East West Management Institute (2015) Fifteenth Quarterly Report USAID (2011) Award Modification 1 USAID (2011) Award Modification 2 USAID (2015) Award Modification 3 USAID (2015) Award Modification 4 USAID (2011) Cooperative Agreement AID-112-A-11-00002 East West Management Institute (2015) Memorandum: Impact of SEDA’s Relations with GOAJ Counterparts East West Management Institute (2015) SEDA General Full Presentation East West Management Institute SEDA 14th Quarterly Report Annex 3 East West Management Institute (2015) Revision of SEDA PMP- Justifications and Rationale for Changes East West Management Institute (2015) PMP With Both Actuals East West Management Institute Post-Training Evaluations of CDCs