January 2013 Partners in Development Services (PDS) USAID/Sudan: dg fixed obligation grants (FOGs) Evaluation report Report Submitted to USAID/Sudan’s Democracy and Governance Office Khartoum, Sudan January 7, 2013 Page 2 Acknowledgement PDS is a Sudanese Consultancy Firm with a main focus on research and training (Annex 6). During 2012 PDS was assigned by USAID/OFDA, AECOM, UNDP, JICA and the World Bank to assess and evaluate some of the projects for which they had provided funds. The evaluation team owes the largest debts to all Fixed Obligation Grants (FOGs) recipients, their respective implementing partners and USAID Democracy and Governance (USAID DG) staff who agreed to meet us with short notice and in some cases during the EID vacations and allowed us to ask sometimes uncomfortable questions. The names of each are included in Appendix (4): People met at the end of this document. The team has also had the unique advantage of benefiting from the insights of stakeholders and USAID-DG staff who attended the debriefing sessions. The team expresses gratitude to Partners in Development Services (PDS - Annex 6) support personnel for their invaluable contribution and in particular during the analysis and conclusions contained in this report. Page 3 Table of Contents Page List of Acronyms 4 1. Background and Purpose 7 2. Study Approach and Methodology 7 2 Findings 8 2.1 Relevance of the project activities to context 8 2.2 Measuring the program results of the completed FOGs 9 2.2.1. Contribute to Educating Sudanese People on constitution making 9 2.2.2. Support the Role of Women in Promoting Democratic Values 12 2.2.3. Building the Capacity of Selected Civil Society Organizations 14 2.3 Tracking Progress of the Active FOGs 15 2.4 Measuring the Overall Effectiveness of the FOGS as Programming Mechanism 18 Conclusion 21 Recommendations 23 Page 4 Acronyms AO Agreement Officer AUW Ahfad University for Women CBOs Community-Based Organizations COR Contracting Officer Representative CPA Comprehensive Peace Agreement CSOs Civil Society Organizations EU European Union FOGs Fixed Obligation Grants GJD Governing Justly and Democratically HAC Humanitarian Aid Commission IDCS Institute for Development of Civil Society IDPs Internally Displaced Persons INGOs International Non-Governmental Organizations NCF National Civic Forum NED National Endowment for Democracy NGOs Non-Governmental Organizations NORAD Norwegian Organization for Relief and Development PDS Partners in Development Services REED Rural Extension, Education and Development, RIG/PDR Regional Institute for Gender, Peace, Diversity and Rights SCDD Sudanese Centre for Democracy and Development SCLS Sudan Comparative Law Society SICM Sudanese Initiative for Constitution-Making ToT Training-of-Trainers UNDP United Nations Development Programme UNFPA United Nations Population Fund USAID United States Agency for International Development USAID DG United States Agency for International Development’s Democracy and Governance Office USIP United States Institute for Peace Page 5 Executive Summary: Background: This report presents the purpose, findings and conclusions of an evaluation carried out by Partners in Development Services (PDS). PDS was contracted by USAID to evaluate the performance and impact of three completed projects, track the performance of two active projects and observe the status of a newly awarded grant. Since June 2011, the USAID-DG office funded under the newly established Fixed Obligation Grants (FOGs) mechanism, six projects in Sudan with a total of $1,297,672. All the projects were/are implemented by four Sudanese civil society organizations (CSOs). The main focus of the projects was to empower the Sudanese CSOs and to enable them to play a role in determining the future of their country and people. They addressed a wide range of issues such as human rights, gender, capacity building, rule of law and good governance, democracy, transparency, accountability and public participation which all are highly relevant to the present socio-political context in Sudan. In general, the funded projects deal with two of the program areas of GJD (governing justly and democratically). The first area is Rule of Law which aims at advancing and protecting human and individual rights and the second area is Civil Society which focuses on empowering individuals to peacefully exercise their rights of expression, association, and assembly. The projects are monitored by the Khartoum-based USAID-DG technical officers who developed mechanisms and strategies for assessing the performance of these projects by collecting and reviewing performance reports from implementing partners and conducting site visits. Funded projects are demand driven and their project documents were designed in a manner that would ensure achievement of the overall objectives. The evaluation team examined the project documents and concluded that they are appropriate and relevant to Sudan’s contextual realities. The Evaluation: The purpose of this evaluation is to better understand, not only how the DG FOGs have performed to-date, but also how effective FOGs are as a programming mechanism for DG activities in Sudan. Areas of focus for this evaluation include, but are not limited to, what USAID and partners are obtaining from the FOGs, what is going well and what is not, and how lessons learnt can be applied to future programming. The end result of this exercise will serve the following purposes: 1. measure program results for completed FOGs; 2. track progress of active FOGs; 3. provide lessons learned for USAID, assessing the potential for key successes to be replicated; and 4. measure the overall effectiveness of the FOGs as a programming mechanism with respect to DG’s overall strategy in Sudan and in light of management burden and technical staff capacity. This report assesses the extent to which each of the completed FOGs met its overall goal; how realistic and appropriate the design of the projects was; whether the appropriate stakeholders were involved in the program and to what extent the program could promote better coordination and collaboration between them? The evaluation team used a number of methods that included review and analysis of project documents, reports, agreements and other relevant documents, This is in addition to collection of Page 6 quantitative and qualitative data, focus group discussions, semi-structured interviews, questionnaires and views and inputs of critical stakeholders. Given the hostile and difficult environment, the wide geographical area of operation and the limited human and financial resources of the implementing partners, the evaluation team feel confident to argue that the projects had managed to achieve the stated milestones. However, the attempt of assessing the extent to which each FOG met its overall goal and analyzing the internal validity of the projects proved challenging due to the lack of a parameter to measure against before the intervention or having the opportunity to spatial equivalence between treatment and control groups. However, they are strong indications of positive impact, in particular, with regards to the forthcoming constitution making related projects that experienced wider participation and dialogue compared to former Sudanese experiences in the same field. The Projects’ Design: As indicated earlier, the design of the projects is realistic and appropriate. The recipients proved capable of generating a huge multiplier effect. Their membership/implementing partners consist of some 40-70 NGOs as in the case of SICM and NCF, or having the enabling environment and suitable constituency as in the case of the AUW, or the credibility and respect of the targeted groups and authorities as in the case of the SCLS. USAID DG, therefore, had succeeded in identifying CSO partners with a proven capacity in relation to achieving milestones/outcomes. In relation to appropriateness, application, awarding and administration procedures of FOGs as a mechanism for DG programming, all the recipients stated that the mechanism proved effective, and appropriate. They believed it to be time-saving and having specific milestones to deliver. This, they argued, helps achieving outcomes. In general, FOG’s pros outweighed by far the cons. Recommendations: The evaluation came up with two sets of recommendations. The first addresses USAID whereas the second addresses FOG recipients. The first set stresses on the continued use of FOGs as a DG programming mechanism; continued support to democracy advocates in Sudan and providing funding to some of the current projects for at least two years. In particular, to the capacity building efforts of projects that focus mainly on women and youth. USAID DG is also encouraged to consider funding for other GJD areas and in particular key elements of good governance such as legislative functions and processes, local government and decentralization, and anticorruption reforms. FOG’s recipients call for more concerted networking between actors, engagement with political parties and media; opting to use of communication channels that appeal to youth and women; and developing benchmarks or other measures to integrate women as important allies in constitution-making. Page 7 Main Part of the Report 1. Background and Purpose The purpose of this assignment is to (i) evaluate three completed DG FOGs, (ii) track progress of two active FOGs and (iii) report on the status of one newly awarded project. The main focus, however, will be on the completed FOGs. This report assesses the extent to which each of the completed FOGs met its overall goal; how realistic and appropriate was the design of the projects; whether the appropriate stakeholders were involved in the program and to what extent did the program could promote better coordination and collaboration between them?; The report also assesses how effective FOGs are as a mechanism for DG programming in respect to DG’s overall strategy in Sudan and in light of management burdens and technical staff capacity. The evaluation was guided by the USAID’s standards; objectives of the USAID FOGs and the FY 2012-2014 Strategic Framework for Sudan; questions raised by USAID in the evaluation terms of reference (ToR); and FOGs recipients' strategies. In addition, the team developed and used performance and effectiveness indicators in line with the log-frames or project documents. The team also developed checklists: and considered relevance, efficiency, value added for USAID/CSOs, lessons learned and case studies for success, short falls, and failure stories. 2. Study Approach and Methodology The evaluation process was grounded in a comprehensive and standardized methodological approach (Annex 1) based on USAID’s guiding policies, principles and criteria for similar evaluations. This involved collection of primary and secondary data. Secondary data collection was intended to improve and shape understanding of the whole exercise. In this respect the following sources were consulted and reviewed: USAID policy documents (FOGs/USAID Forward etc.), original unsolicited proposal for each FOG, grant documents and work plans, and the final/evaluation reports for each of the completed FOGs. Primary data constituted the core of the analysis and the evaluation process. The data was collected through direct fieldwork conducted in 9 States namely: Red Sea, Khartoum, River Nile, Kassala, Gedarif, South Kordofan, South Darfur, Gezira and Blue Nile (Annex 2). The distribution of these States through the presence of SICM, NCF, SCLS and AUW is provided in the table below. KRT R. Nile R. Sea Kassala Gedarif B. Nile S. Kor S. Dar Gezira SICM          NCF       AUW     SCLS  The selection of the 9 states took into consideration the following: Enabling and disabling contexts in relation to security issues and constraints. States with perceived higher and lower achievements as maintained by previous evaluations of completed FOGs. Accessibility Page 8 Innovative approaches adopted in specific states that could provide for future learning The field work was based on a participatory and cross learning process using the following tools:  Checklist: A check list (Annex 3) capturing the four key questions and other questions on relevance of activities to context; role of improvement (How) in relation to main programme thematic areas; identification of stakeholders, capacity needs, people empowerment (political – socio-cultural and other issues of rights) was prepared and shared with USAID. Comments received from USAID were incorporated to refine the final check list. The relevance of the check list to the interviewees was seriously considered.  Focus Group discussion: A group discussion was held in specific with M.Sc. students who benefited from Ahfad University scholarships with the objective of capturing their own assessment of the USAID funded projects. A group discussion was also held with the SICM Coordination Committee..  Semi structured interviews with a wide range of stakeholders and key resource persons (Annex 4).  Interviews with USAID DG, recipients and implementing partners.  Debriefing: The evaluation team held a debriefing meeting and shared its findings and conclusions with USAID and the recipients of FOGS (Annex 5). Limitations:  The deteriorating security situation in South Kordofan as a result of the fighting between Sudan Armed Forces (SAF) and the SPLM/A-North had left SICM with the only option of shifting their work to Khartoum. As a result, the field work of SICM-South Kordofan was conducted in Khartoum where interviews were held with SICM partner CSOs currently based in Khartoum. With reference to NCF training activities were shifted from South to North Kordofan State where the interview was conducted in collaboration with El Obied Voluntary organization which acts as a liaison and logistic partner to NCF.  The field work was conducted immediately after the Eid Holidays at a time when some of the leaders of the targeted CSOs were still at their home areas to spend the Eid holidays with their families. However, the evaluation team had managed to interview them in Khartoum, Gedarif and Kassala. 2. Findings: 2.1. Relevance of the project activities to context In general, the funded projects deal with two of the program areas of GJD (governing justly and democratically). The first area is the rule of law that aims at advancing and protecting human and individual rights. The second is civil society which focuses on empowering individuals to exercise peacefully their rights of expression, association, and assembly. Page 9 Examining the activities of each FOG, reiterates their relevance to the present Sudanese contexts as detailed below: The activities of the SCLS project are high on the agenda of the human rights activists. Judicial independence and justice systems are key elements of the role of law. Appropriateness and relevance of key activities is examined below: on the national level, they seek engaging all members of the legal profession (judges, practicing lawyers, legal advisors of the Court, law Professors and students) in serious discussions about the legal system in the Sudan. It also aims at reaching members of parliament and policy makers, on the pressing issue of law reform through its varied program of lectures, seminars, workshops, group discussions and publications; it also aims to reach Sudanese lawyers within and outside the Sudan. Many of its activities will also be open to ordinary lay men in order to enhance their legal knowledge. It also aims to collaborate fully with civil society organizations in promoting the principles of good governance, the Rule of Law and the protection of fundamental rights in the Sudan. Should the two-year plan be implemented, then this will no doubt be a yardstick for gauging the activities of the Sudan Comparative Law society and the success of its efforts in promoting the cause of the Rule of Law and justice in Sudanese Society. The activities of the completed SICM project aim at preparing the Sudanese people to participate actively and effectively in the forthcoming constitution-making process through a nationwide civic education and public awareness campaign. A task highly required in Sudan where few elites and politicians were involved in the constitution drafting excluding the masses of the population and in particular women, youth and the marginalized groups such as the IDPs, urban poor and persons with disabilities. The activities of the AUW active project aim at coordinating the varied initiatives and efforts of the civil society sector for the upcoming constitutional review by bridging the gap of information’s non-existence, establishing more structured and consistent networking, sharing and exchanging of information and lessons learned and documentation of the process and hence the project advocate for the principles of participation, inclusion and transparency.. The NCF deals with empowering individuals to exercise peacefully their rights of expression, association, and assembly, including through their establishing and participating in NGOs, unions, and other civil society organizations 2.2.Measuring the programme results of the completed FOGs 2.2.1 Institute for the Development of Civil Society (IDCS) in collaboration with the Sudan Initiative for the Constitution Making Process (SICM): “To promote a nation-wide debate on the development of Sudan’s permanent constitution following the end of the Comprehensive Peace Agreement (CPA)”. Project Period was June 21, 2011 to September 20, 2011 with a fund of $ 99,997. The project included three outcomes which are as follows: 1) Training of 60 trainers/facilitators ; 2) Developing of a civic education curriculum and advocacy material; and Page10 3) Conducting 90 forums (6 in each of the 15 states). All of the above results were achieved. The training of 68 participants was successfully completed, one of the ToT training workshops was held in Khartoum in July 2011; a training manual, educational and promotional material were prepared, pretested, published and circulated to the 15 states; and 75 forums were organized (6 in each state including Khartoum). That means SICM was able to organize an additional number of forums in collaboration with other CSOs and local authorities. In total, 4,232 people attended these forums. However, it is to be noted that some of the activities, namely the development of the civic education curriculum, advocacy material and forums were delivered after the dates stipulated in the grant’s document. The overall goal for the project was to “Prepare the Sudanese people to participate actively and effectively in the forthcoming constitution-making process through nationwide civic education and public awareness campaign”. The extent to which actual results contributed to achieving this overall goal is discussed below: There were many challenges and constrains which hindered the implementation of the project. In addition, there were opportunities for improvement in relation to the main thematic areas, identification of stakeholders, extent of participation, capacity needs and how people are empowered. The information collected through the review of SICM’s reports and interviews with SICM staff and focal points revealed that SICM had a wide geographical coverage as it had conducted forums and workshops in all the 15 states. However, coverage within the states had, in most cases, focused on state capitals for two main reasons: (i) 90% of the trainees were chosen from the state capitals to the extent that some localities were not represented. This fact posed a serious problem as the forums were conducted in the state capitals and the surrounding localities. The evaluation team noted that SICM’s project proposal did not mention where the forums should be held. The proposal mentions only the number of forums to be organized in each state. Since SICM’s intent was to develop a cadre of focal points at each state, it is natural that they had focused on state capitals. However, the evaluation team feels that issues of access to remote localities should be considered in future. (ii) Reaching localities and in particular the distant ones requires additional funding taking into consideration the flat rate fund per a forum and a workshop. Many of the trained facilitators did not take part in the state-level activities. To quote examples: only one trainer in South Darfur was expected to take part, in collaboration with one focal point, to cover the 21 localities of the state. In the Blue Nile State, one focal point was appointed a minister in the state (government) and another focal point had joined the SPLM-(N). Although the movement of these individuals had its impact on the activities of the project, it has a positive effect as they would sow the seeds of constitutional awareness elsewhere. The creation of two new states (Eastern and Central Darfur) meant that four states had to be covered by the focal points who were trained to facilitate SICM’s work in the two old states (South and Western Darfur).The inability to retrain some of the focal points is attributed to the fact that the focal points were involved on the basis of three-months renewable contracts. In most states the focal points were found to be active only when they receive funds required for implementing specific activities. In the event that SICM and USAID find it appropriate to extend the project for another period, improving the terms of Page11 engagement of the focal points and strengthening SICM’s relationship with the focal points is a matter of high importance and a fundamental factor for achieving the overall objectives of its project. In this context, it might be appropriate to consider serious options of alternative outreach instrument such as producing CDs, theatrical improvisation DVDs and utilizing community radio in the localities that are difficult to access. In South Darfur, for example, a forum was devoted to the “hakamat” (women poets) who are reputed for instigating war, and the Masheesh theatrical group. Within the Darfur and Kordofan rural contexts, it is easier and more practical to disseminate messages to local communities through such methods and instruments. Unfortunately, due to lack of funding perhaps, no subsequent effort was exerted to utilize the acquired knowledge in advocating for the principles of the constitution-making process. The project document indicates clearly that the new project is meant to pick up where the first one left off. In this regard, the team finds it appropriate to report on the excellent initiative of the focal points in Gazira State who managed to provide translation into the sign language for some workshops organized for the disabled members of society. . The evaluation team finds it difficult to confirm that there is a real partnership between the focal points and SICM. However, the team took note of a very positive development in Gazira State where the focal points made the initiative of establishing a network of local CSOs that share interest in the constitution-making process. The representatives of the Sudanese Centre for Democracy and Development (SCDD), one of SICM’s partners in Wad Madani (Gezira State), informed the evaluation team that when they are in the process of conducting a forum or a workshop, the other members of the network usually send their volunteers to assist the Centre. The present organizational relationship between SICM and its focal points is fragile as the latter believe their role as implementing agents only when instructed by SICM’s headquarter in Khartoum. In almost all other states there is the risk of the focal points pursuing their own personal agenda? In some instances, according to SICM State and Training Coordinator, the selection of the trainees was limited to family members. According to SICM’s leadership this had happened in one state and they managed to avoid repeating it in other states. Currently, SICM is assessing the work of its focal points in an effort to address problems encountered. The attitude of the official state and national authorities vis-à-vis the SICM network and its activities under this project was found to be inconsistent and contradictory. It ranged from the negative reaction of the security authorities by hitting hard on the focal points at different levels to the positive approach of requesting coordination and collaboration, as in the cases of the Advisory Council on Human Rights and some State Legislative Assemblies. The hostile attitude of the National Intelligence and Security Services (NISS) has not selectively targeted projects funded by USAID FOGs or their implementing CSOs. In an article published by the Sudanese electronic newspaper, Dr. Amin Makki Madani, a highly respected legal expert and human rights activist, expressed the view that the NISS has for some time targeted all the Sudanese civil society organizations (CSOs) that advocate for social, economic, cultural and political rights; which means all those supporting democratic transformation, the rule of law, human rights and basic freedoms. According to Madani, the Humanitarian Affairs Commission (HAC), which is currently under the framework of the Ministry of Interior (not the Ministry of Social Affairs), and NISS had recently carried out investigations with a number of non-governmental organizations such as Alkhatim Adlan Centre for Enlightment and the Sudanese Writers Union. The latter was investigated for organizing a debate forum on the forthcoming constitution (Funding of Civil Society Organizations, www.sudanile.com, 21 November 2012). Page12 In relation to this project, the evaluation team has reached the following additional conclusions: The evaluation team is of the view that although the project achieved all stated results, it did not fully meet its overall objective which was very difficult to achieve by one project that is expected to cover 15 states in one year. The reason is that the overall objective was too optimistic especially when it relates to “preparing the Sudanese people to participate actively and effectively in the constitution-making process”. However, the evaluation team believes that given the difficult and complex political environment, the project has contributed significantly in exposing to the public the importance of the constitution-making process which was considered in the past as the task of a selected group of elites and could only be done by officials. As a result, constitutional issues have become a part of public debates. The Council of States (the second chamber of the national parliament), the Consultative Council on Human Rights, State Legislative Assemblies, trade unions, youth and women organizations requested SICM in Khartoum and State capitals to organize forums for them on the constitution-making process. Equally important, the training manual and promotional materials (posters, pamphlets etc.) which were prepared by SICM are now used by a number of organizations and other initiatives on the constitution-making process. According to the focal points interviewed by the evaluation team, SICM’s messages and promotional material targeting the public are impartial and of technical nature, hence, do not cause provocation of any nature.. In Al Gezira State, for example, the security officer who checked the material stated that he had no objection to promotional material. One of SICM’s key messages which drew the attention of the public to the importance of the constitution-making process was a radio program which discussed household’s daily food basket, its relation to the constitution principles and the need for a bill on socio-economic rights. 2.2.2. Support the Role of Women in Democratic Values – AHFAD UNIVERSITY FOR WOMEN . Project Period: July 19, 2011 to July 20, 2012 with a funding budget of $ 99,264 . The project: The goal of the project was to contribute to enhancing the human resources capacities of Sudanese people for increased empowerment, engagement and awareness. The main objective was to increase awareness about the values and principles of peaceful coexistence, equality, sustainable development, human rights and good governance. The project had two main components. The first was raising the awareness and providing vital skills for AUW students in the fields of democratic values, human rights and good governance and the second was enhancing the capacity of young female academics in marginalized state universities. At the time of the project design and throughout the implementation period, the project was found to be relevant to the context (in terms of target, messages and materials/methods used), appropriate and realistic as all the milestones were achieved successfully. Page13 The Project’s Achievements: Based on reviewing the project document, project’s evaluation report and interviews conducted with AUW director of international relations office, Dean of School of Rural Extension, Education and Development (REED), AUW core group and MSc Students, the evaluation team found that: The Human Rights, Democracy and Governance Reader was developed, tested and published. In addition, 1000 copies of the information, education and communication (IEC) material were produced and disseminated. To serve the purpose of using the Reader as part of the student’s field trips materials and part of the curriculum of REED, the production will be replicated by AUW. A core group of 15 AUW students, representing the undergraduates and post graduates, will be trained to test, evaluate and use the Reader. The AUW commitment to rural women and communities is translated by its unique Rural Extension Program, established in 1973 with the long term goal of empowering rural women, their families and communities, and to mobilize them towards sustainable rural development. The program includes scientific field trips to explore the life of rural people and the means and ways for proper interventions to improve the quality of their life style. In 1976 the rural extension course was integrated into the curriculum of the school of Family Science. One year later the course became a university requirement for all the schools. The key participants in the program activity are the students and faculty members who train students and the rural communities especially women. Other stake holders are government officials and NGOs working in the area. 20 students from the less developed states (South Kordofan and Eastern Sudan States) were selected and joined the M.Sc. programme (15 as Gender and Migration MSc student and 5 as Sustainable Development MSc students). The M. Sc. Students, who are now in the second semester of the programme, are expected to act as supervisors in the forthcoming field trips. A memorandum of understanding (MOU) was signed with 8 states’ Universities in Darfur (El Fasher, Zalinji, and Nyala), Blue Nile and Kassala States to accommodate the M. Sc. students after graduation so as to guarantee the involvement of the students at their communities’ level. The enabling environment in AUW with regards to qualified academic staff, large number and diversified groups of students, the project’s staff and M.Sc. students all contributed to the success of the project. The second component of the project was encountered by the challenge of selecting the M. Sc. students. The essence of this component is to reach the teaching assistants of the States’ Universities who lack access and opportunities for training. Due to difficulties related to establishing contacts with the said universities at the time of selection, AUW opened the scholarships for female students from the marginalized states who had graduated with a basic degree of grade 1 or 2 (distinction or very good). To avoid encountering similar problems in future, AUW is advised to publish advertisements in the local newspapers inviting the concerned universities to nominate their candidates for these scholarships. The outcome of the interviews conducted with 10 (out of 20) M. Sc. Students suggests that there is no guarantee that they intend to join the marginalized states’ universities after the completion of their studies. Instead, they were found to prefer working with other employers such as NGOs/INGOs. The sustainability of project impact is guaranteed through: Page14 The well-established academic networking and relations with the universities in the marginalized states. The M.Sc. graduates are expected to act as change agents among their communities, whether they join the state universities or work for NGOs/CBOs. The Human Rights, Democracy and Governance Reader is the strongest element of the project’s sustainability as it will be used in multi-dimensional ways; in the REED curriculum and in the annual field trips material/ messages. The AUW had approached the French Embassy to fund the reproduction of the Human Rights, Democracy and Governance Reader and to train youth groups composed of 30 students each to use the reader for their future activities (as a youth groups or as members of CBOs). In an effort to ensure the sustainability of the post-graduate scholarships, in case the USAID may decide to stop its support, AUW had approached other donors such as the Sudan American Foundation, NORAD, UNFPA and Micro-Peace Building Project. Internal Validity: The project succeeded in introducing new concepts of democracy and governance to large numbers of beneficiaries. The list of beneficiaries includes:  AUW students (7000).  Outreach to grass root communities: the reader became part of the university’s curriculum, field trips’ material and outreach activities. This year, the university’s field trips started on the 15th of November, and reached out to 70 rural areas with the participation of 1,050 undergraduate students under partial supervision from the M.Sc. students funded by USAID. USAID scholarships provided great educational opportunities for 20 M. Sc. Students: 15 in the newly introduced field/paradigm of gender and migration and 5 in the Sustainable Development program. The project helped build the capacities of the students who will use the knowledge and skills they gained in universities and NGOs/CBOs that they would be join after their graduation. The following comment by one of the students who was interviewed by the evaluation team is self￾explanatory and explains the project’s added value. “Although my M. Sc. Degree will be awarded by the university, I think without the USAID scholarship it could not have been possible for me to pursue post-graduate studies.” 2.2.3. Building the Capacity of Selected Civil Society Organizations in Eastern Sudan, Blue Nile and South Kordofan – National Civic Forum (NCF) - Project Period: August 28, 2011 to February 28, 2012 with a budget of $99,652. The overall objective of the NCF project is “To strengthen the capacity of selected civil society organizations”. NCF’s specific program activities under this grant were: A needs assessment survey; 10 training workshops: three on NGO management, three on budgeting, proposal writing and reporting and four on advocacy campaigning and mobilization. Page15 A training workshop on monitoring State Assembly performance; Dialogue forums (one in each State), and Six public lectures on democratic transformation, socio-economic rights and civil society roles. Tracing achieved results were also linked to the implementing organization’s evaluation report that bluntly stated: “There is evidence that NCF managed to fully implement the needs assessment survey; 9 out of the 10 training workshops; the training workshop on monitoring the performance of state parliaments; and 6 public lectures on democratic transformation, socio-economic rights and civil society.” For the purposes of evaluating this project, the evaluation team reviewed the project document, some of the project’s progress reports and the project evaluation report which was prepared by the implementing USAID partner (NCF). The team had conducted interviews with the NCF chairperson, two of its partner organizations in Kassala (Delta and the Union of Persons with Disability) and El Obied Voluntary Association in North Kordofan. Based on information collected through this process, the evaluation team came up with the following conclusions: The training workshops contributed significantly to building the capacities of the CSOs in the five states, specifically in relation to coordination and networking. A group of skilled trainers in the area of civic education was created in the five states. The workshops enhanced the NGOs/CSOs capacity to campaign and advocate for democratic transformation, public accountability and socio-economic rights. Networking and exchange of experience among NCF partners in different states was strengthened. The project design was very appropriate and facilitated the monitoring and evaluation task despite the disenabling political environment. Elements of sustainability were maintained, in addition to creating a pool of trainers and training material by securing funds from the Arab NGO Network to fund engagement of the trained trainers in training other CSOs in their respective states. However, the evaluation team failed to realize the role of NCF implementing partners beyond the logistical preparation. Taking part in selection of the participants was limited to those coming from Kassala and North Kodofan. It is difficult to assume that delivering a training workshop on monitoring the performance of the state legislative assemblies in Kassala (for Kassala and Gedarif States) would mean that a model for CSOs monitoring of parliamentary work in the two states had been initiated. 2.3. Tracking progress of the Active FOGs: 2.3.1. Constitution Building Project- Ahfad University for Women - The Regional Institute of Gender, Diversity, Peace and Rights: Project period: March 23, 2012- March 27, 2013 with a funding budget of $499,476. The goal of the project is to coordinate the varied initiatives and efforts of the civil society sector for the upcoming constitutional review by bridging the gap of information, establish structured and consistent networking, sharing and exchanging of information and lessons learned and documentation of the process. The project has four main components; the establishment of and information hub on constitution/human rights/legal issues, mapping survey and documentation, Page16 capacity building on critical analysis and assessment of legislations and exchanging experiences on constitution context. At the time of design, the project was very realistic and aims at achieving a couple of objectives: Raising awareness on the constitution building process and human rights. Educating the public and policy makers on some important issues such as diversity, citizenship rights and decentralization of powers (federalism). Broadcast the concept of public participation in building our social contract which will contribute to the state of sustainable peace, democracy and development. Offer a space to discuss and come to agreement/options about elements of the constitution. Building the capacity of stakeholders in constitution building issues. Achievements: Four capacity building workshops/state-based dialogues in Khartoum, Gezira, Gedarif and North Kodofan. The papers presented and the groups’ recommendations were amalgamated into a report which was distributed to the participants in the Legal Analysis Course. . A two weeks training course on legal analysis and one training course for curriculum development. One ToT program for 15 future trainers in the field of constitution building and legal analysis. A Constitution Development Manual developed and produced. Mapping of entities active in civic education on constitutional issues covered 965 entities in 15 states. An effective link with universities and peace centers in Darfur, South and North Kordofan, Gedarif, Blue Nile and White Nile. For the website, the contracting step with the website’s company is completed and data entry is under process. Creation of face book group ( ) to make use of the social media space. Establishment of a coordination committee representing the 12 universities aiming to reach a consensus on the new well-articulated constitution. To ensure the involvement of the appropriate stakeholders, the project incorporated the experience and participation of legislative bodies, executive boards, youth, and women’s groups, political parties in collaboration with national and international experts, academicians and project’s staff. The project activities are progressing well and on the right track. The evaluation team has the impression that the AUW has the capacity and commitment to complete the implementation of this project in time and as stated in the project document. The survey encountered by serious challenges with the security authorities in some states. This has resulted in limited access to some entities. To overcome this problem, AUW is planning to upload the questionnaire in its website so that it can be accessible to all entities that missed the opportunity during the time of the survey. At the beginning, the AUW faced serious challenges in developing and managing the constitution’s website due to the difficulty of recruiting technical personnel who have good IT skills or contracting a reliable and efficient company to do the business. Eventually, however, the AUW managed to recruit skilled personnel who have the capacity to develop the website. At the time of field work, the evaluation team noted that a considerable progress had been achieved in developing the website. Page17 The sustainability of the project impact could be guaranteed through: Sustainability of the website. Ongoing courses; curriculum at both levels of the B. Sc. and M. Sc. Replication of the training workshop by states’ universities. Continuous public lectures within AUW. Part of the AUW M.Sc. program was to attach each student to one of the organizations working in the field of constitution-making. Therefore, the first AUW project on empowering women compliments and serves the objectives of the second project on constitution-making. Working in collaboration with the local media outlets. A specialized manual on women’s constitutional rights 2.3.2. Sudan Comparative Law Society (SCLS) - Project period from April 18, 2012 to April 14, 2014, with a fund of $99,648. Project’s objectives are: To explain and evaluate current legislations with reference, in alia to legislations and other similar situations. To report and comment on current judicial precedents, including judgment of the constitutional court. To exercise continuous surveillance of the application of the rule of law. To undertake and encourage legal research, with emphasis on the comparative based research; To promote law-abiding spirit and public awareness of the duty to abide by the law. To undertake and encourage the research is the Sudan customary laws. To undertake researches relating to law reform. To undertake continuous evaluation of the curricula of Sudan Law schools and legal training programs. SCLS intends to achieve the above objective through activities that include conducting studies and research, workshop, seminars and group discussions, public lectures, training, and establishment of legal data base in addition to curriculum development of law schools, publishing of a periodical, translation and exchange visits. Achievements: The only milestone achieved so far is related to staffing and furnishing the SCLS office in preparation for activity implementation1. A workshop on “Imposed By-Laws that are not Endorsed by the Parliament” is scheduled to take place on December 24, 2012, in addition SCLS is organizing for a forum and a lecture on two major constitution issues namely independence of the judiciary and power and wealth sharing. 1 SCLS subsequently organized its first forum under the theme “Key Issues for the Future Constitution.” Upon an AOR recommendation, the AO issued a letter reminding SLCS to speed up implementation and offering to revise the benchmark calendar if deemed necessary. Page18 SCLS members are well reputed and highly experienced professionals; they all are known for their integrity and neutrality. However, 5 out of the 11 members (around 45.5%) of the Executive Committee are above 71 years of age. Many of the new SCLS members are young and prominent community figures who, even though they add positively to the image of SCLS, are not empowered by the SCLS elders. This factor led to slow and limited deliverables in the first six months of this two-year project. To overcome this constraint a number of propositions were discussed separately with the SCLS Secretary General and a member of the Executive Committee which are as follows: Injecting new blood into SCLS. Young staff members of law schools who believe in principles of Rule of Law and Human Rights should be head-hunted and mobilized. Establishing operational committees composed of the above mentioned groups and delegating them to carry out specific tasks; Initiate a web-page and make use of the social media to secure participation of wider groups of public in relation to by-laws. 2.3.3. Popular Sovereignty and the New Constitution (IDCS/SICM, US$ 399,635 - 30 September 2012-29 September 2013): The grant for the IDCS/SICM popular sovereignty and the new constitution project (30 September 2012 – 29 September 2013) was just approved at the time of the evaluation. Therefore, it was not possible for the evaluation team to report on progress in implementation. 2.4. Measuring the overall effectiveness of the FOGs as a programming mechanism FOGs Objectives, Guidelines and Requirements Recipient Feedback FOG format allows performance without monitoring the actual costs incurred by the recipient All recipients informed that the FOGs system is a time-saving funding/programming mechanism and facilitates implementation and monitoring. USAID pays the recipient a set amount when it accomplishes a milestone. FOGs recipient confirmed receiving the agreed amount following achievement of a milestone. USAID must assess the risk factors that could prevent the proposed recipient from completing the activity or require a substantial change in milestones (risks are assessed and mitigated). The feedback from USAID FOGs staff and the recipient CSOs suggests that project proposals were reviewed thoroughly and critically before awarding grants. For non-US NGOs, the total value for each year of the FOG must not exceed $ 500,000. The amounts of grants awarded to the Sudanese CSOs range from a minimum of US$ 99,264 to a maximum of US$ 499,476 (both were awarded to AUW). The duration of the FOG must not exceed three years. The duration of the six projects under evaluation is 3- 24 months. The AO may amend milestones during No AO amendments were made to scope or Page19 period of grant, if original milestones no longer feasible or appropriate due to circumstances beyond the control of recipient, and if amended milestones compatible with and satisfy original purpose of grant. deliverables of any of the grants. USAID, however, allowed some flexibility in achieving some of the deliverables, such as in the selection of the students in the Ahfad university’s M.Sc. scholarship program. While students were from the marginalized states specified in the project document, they were not all staff members of the universities in those areas as initially planned. The recipient can fulfill the objectives of the grant Both the USAID FOGs staff and the recipient CSOs provided information confirming that the integrity and capacity of the recipient organizations to deliver and fulfill the objectives of the grant were reviewed and assessed before the award of the grant. USAID can close out the grant and evaluate the results. FOGs recipient made their own evaluation for the three completed projects and provided copies of their evaluation reports to USAID. Following the submission of this evaluation report USAID can easily close out the completed grants. Simplify the USAID procedures for award of FOG. Recipients of USAID FOGs informed that compared to the funding mechanisms of other donors such as the EU and UNDP, FOGs mechanism is simple, time￾saving and flexible. Provide for technical assistance to recipients, to strengthen them as USAID partners. The grant recipient informed that unlike the funding mechanisms of other donors, FOGs can benefit small and newly established CSOs. The grant awarded to the Sudan Comparative Law Society (SCLS) is an evidence of an effort that aims at strengthening USAID partners. The positive feedback from the four CSOs which were awarded FOGs is consistent with and supports one of the main objectives of the USAID implementation and procurement reform plan which is “to strengthen local civil society and private sector capacity to improve aid effectiveness and sustainability” (USAID Forward, P. 6). The views of the recipient organizations on the overall effectiveness of FOGs as a programming mechanism can be summarized as follows: The FOGs mechanism is the best if compared with the bureaucratic and time-consuming funding requirements of other donors such as the EU and UNDP. The FOGs mechanism which is based on specific milestones and time-frames facilitates planning, implementation, monitoring, evaluation and liquidation processes. FOGs is a flexible mechanism as it allows for implementing more than one milestone at the same time without having to follow the sequence (matrix) of activities as stated in the project document. . The mechanism is very flexible and responsive to the circumstances and unexpected developments on the ground. For example, when the National Civic Forum (NCF) realized that, for security reasons, it was not possible to organize training workshops in South Kordofan State, it contacted one of its CSO partners to assist in conducting the workshops in Page20 North Kordofan State. The NCF partner managed to host the workshops which were attended by participants representing CSOs in South and North Kordofan States. Small and newly established NGOs/CSOs such as the Sudan Comparative Law Society (SCLS) can benefit from the FOGs funding mechanism. FOGs support will help building and strengthening the capacity of the SCLS. It is very difficult for CSOs like SCLS to receive funding from other international donors in Sudan. Linking release of grant funds to achieving milestones ensures financial accountability and, therefore, minimizes the possibility of corruption and abuse of project funds. Furthermore, it is seen by the recipients as a tool for organizing their work and activities. FOGs mechanism requires a brief and simple concept note/proposal. USAID FOGs staff follows up closely the other activities implemented by their partners. For example, when the newspapers published information on NCF work on environment, NCF received a message of congratulations from USAID. FOGs staff develop an effective and smooth system of communication with their partners. USAID staffs overseeing FOGs are deeply involved in all the steps of project implementation. USAID staff overseeing FOGs are well engaged with the Sudanese civil society especially on issues of democracy and governance unlike the UNDP which deals only with the government and perceives civil society as a complementary component. FOGs mechanism requires a very brief report on each milestone achieved. The mechanism is better than those of the other international donors in terms of the quick response to project proposals. The evaluation team noted that a maximum period of one month, after the submission of the final proposal, is required for issuing a letter of award and a contract for the recipient CSO. The mechanism is very good as the payments are made in time and as agreed. Gender equality is a prerequisite in the project design, implementation, activities and evaluation. The above summary of the views of the grant recipient CSOs are relevant and consistent with the USAID strategic objective which states that “our goal at USAID is to create the conditions whereby, one day, aid will no longer be necessary in the countries where we work. That goal can only be realized if the assistance we deliver strengthens the local actors and institutions that are ultimately responsible for transforming their countries” (USAID Forward: Implementation and Procurement Reform, (P. 6). It is also consistent with and serves the purpose of the USAID Strategic Framework for Sudan (FY 2012-2014) when it informs that “this strategy framework focuses on conflict mitigation, governance and the transformation of humanitarian relief to development and builds upon lessons learned about how such programming can be carried out most effectively” p (5). Moreover, the recipient feedback is in line with the USAID operating principles of promotion of gender equality and female empowerment, measurement and evaluation of impact, building-in sustainability (where appropriate) from the start, application of integrated approaches to development, and creation of strategic partnerships (p. 8) Two out of the four recipients (SICM and NCF) considered the security situation in Khartoum, South Kordofan, Blue Nile and Darfur as the main factor which had negatively impacted the Page21 implementation of their activities. The security restrictions had also limited the movements of the USAID staff to attend and monitor some activities implemented by their partners. 2.4. Conclusions SICM completed project achieved all stated results but did not meet fully its overall objective which was very difficult to achieve due to the large geographical area it planned to cover (more than15 states). Another reason is that the overall objective was very optimistic especially when it relates to “preparing the Sudanese people to participate actively and effectively in the constitution-making process”. However, the evaluation team believes that given the difficult and complex political environment, the project has contributed significantly to making the constitution process as a big issue, which was considered in the past as the task of a selected group of elites, a public as well as an official concern.The intermittent relation between SICM and focal points is vivid; the relation is tangible when an activity is being implemented; otherwise focal points maintain own roles as separate entities. NCF completed project has contributed significantly to empowering and building the capacities of the CSOs in the targeted number of states. Training CSOs in management, proposal writing, and lobbying and advocacy campaigns is a well-deserved service for NGOs/CBOs especially in Sudan marginalized areas. NCF: The evaluation team did not foresee a strong monitoring and follow up mechanism in many cases between NCF and its implementing partners. SCLS: The slow pace of implementation characterizes the decision-making process and implementation. Board members, have, high demand on their effort and time. Delegating younger members and mobilizing others can be considered. AUW is well known for its long history in empowering the Sudanese women through education. There is no doubt that the 20 women who are now enrolled in the M.Sc. program will be empowered by being awarded the M.Sc. degree whether they join the state universities or not. There is, probably, no other institution in Sudan that is well placed and has the potential of empowering women than the AUW. AUW came up with a feasible alternative to the rather passive response of the universities in the targeted areas to nominate some of their staff members to benefit from the M.SC scholarships by nominating individual students from the targeted locations. However, no strong measures to ensure that the nominated students would serve their communities for a specified period of time upon the completion of their study were taken. Security Constraints: Three out of the four USAID partner CSOs have encountered security problems while in the processes of implementing their projects. The three CSOs are SICM, NCF and AUW. The attitude of the official authorities was found to be inconsistent and contradicting ranging from the reaction of the security authorities of hitting hard on the focal points at different levels to the positive approach of requesting coordination and collaboration as in the cases of the Advisory Council on Human Rights and some State Legislative Assemblies. The evaluation team found out that the NISS is targeting all the civil society organizations that are active in the field of democracy, human rights, Page22 rule of law and good governance. However, FOGs recipients preserved in implementing their activities and adapting to challenges encountered. The working environment is hostile and harsh. To mitigate harassment of the security to CSOs, revealing these incidents to the outside world and advocating for the right of SCOs to act as stipulated in the humanitarian instrument, human rights conventions and the interim constitution of Sudan Lessons Learned: Initiatives on engaging the “hakamat” (women poets) in dissemination of information to the public especially in Darfur and Kordofan should be financially supported. It is necessary to adopt innovative approaches. The translation into the sign language in forums organized by SICM for the disabled in Wad Madani (Al Jazeera State) is a good example in this regard. Networking with other CSOs is crucial for the implementation of the project activities. In Al Gezira State the (SCDD) had benefited significantly from the assistance provided by its network member organizations that made their volunteers available for implementation of forums and workshops. Collaboration and coordination with the State authorities helps the smooth implementation of planned activities and facilitates the work of the CSOs. The stakeholders who are expected to benefit from the services implemented by USAID partner organizations should be consulted to ensure their collaboration before finalizing the objectives and milestones of each project. Follow up mechanisms need to be established for workshops and activities that are conducted for purposes of coordination and networking between civil society organizations. Whenever possible media outlets such as the state radios should be considered as an effective means for dissemination of information and messages to the public. The interviewees who had experiences in using local radios and TV estimated that the budget for one forum could be used for conducting two or three radio sessions. Success Stories: IDCS/SICM: The focal points in Al Gezira and Nahr Al Neel (River Nile) states were able to conduct additional workshops and forums without using the project funds as a result of their coordination and collaboration with other civil society organizations and state authorities. NCF: The participation of the HAC Commissioner in North Kordofan State in the training workshops conducted by NCF and its local partners had facilitated for the participant organizations to establish and officially register a network under the name of Basma. Subsequently, the network had implemented a number of good projects. AUW: Through USAID support the AUW has been able to incorporate issues of gender, democracy and human rights in its curriculum for the undergraduate and postgraduate students. Page23 2.5. Recommendations: USAID-DG Make the most of relative donor influence in Sudan by building a unified, strong donor and diplomatic front to press the government to make a commitment to ensure participation of political parties and CSOs in the constitution making process. Encourage FOG recipients’ coordination with political parties and the populations of Darfur, Blue Nile and South Kordofan in order to reach consensus on the principles of inclusion, participation and transparency in the making of the upcoming constitution. Continue use of FOGs as a DG programming mechanism and expand funding to build on current projects to support capacity building and focus on women and youth. Consider funding other GJD areas and in particular key elements of good governance such as legislative functions and processes, local government and decentralization and anticorruption reforms. FOG Recipients More concerted efforts to promote networking between actors, involvement with political parties and media; More effective collaboration with mass media channels in educating the masses and opting to use of other channels of communication that appeal to youth and women including but not limited to use of folk arts, text messaging and rap in the case of addressing urban youth. Develop benchmarks or other measures to integrate women and women’s organizations as important allies in constitution making. . Page24 Annex 1 SCOPE OF WORK USAID/Sudan Office of Democracy & Governance Fixed Obligation Grants I. BACKGROUND This evaluation will focus on Fixed Obligation Grants (FOGs) given to local Sudanese organizations by USAID/Sudan’s Office of Democracy and Governance (DG). The Evaluator will address seven FOGs to Sudanese CSOs in total, two completed ones, three operating and two are currently underway. The evaluation will cover both the performance of the projects funded, as well as an assessment of USAID/Sudan’s experience in using FOGs as a direct funding mechanism to support local civil society organizations (CSOs) in furtherance of DG objectives. As a matter of practicality, the main focus of the evaluation will be on completed grants; however, a key objective of this evaluation is to inform Mission management on the progress of newly-awarded grants and provide guidance on the desirability, feasibility, and scale of future assistance through FOGs. Background information on the FOGs administered by the Democracy and Governance (DG) Office will be provided to the evaluator who has been selected. II. PURPOSE OF THE EVALUATION USAID/Sudan is commissioning this evaluation to better understand not only how several DG FOGs have performed to date, but also how well FOGs are working as a programming mechanism for DG activities in Sudan. Areas of focus for this evaluation include, but are not limited to, what USAID and partners are obtaining from the FOGs, what is going well and what is not, and how lessons can be applied to future programming. USAID/Sudan requires the Evaluation Team to design and implement an evaluation of three completed DG FOGs, and also to observe two additional FOGs that will be underway at the time of the evaluation. The end result of these efforts will serve the following purposes: (1) measure program results for completed FOGs; (2) track progress of active FOGs; (3) provide lessons learned for USAID, assessing the potential for key successes to be replicated; and (4) measure the overall effectiveness of FOGs as a programming mechanism with respect to DG’s overall strategy in Sudan and in light of management burden and technical staff capacity. The Contractor shall discuss and analyze program performance and success, and shall also address opportunities missed or accomplishments that fell short of potential or expectations, as these findings could inform future USAID/Sudan grant-making processes, particularly as the Mission’s development strategy for Sudan evolves. As USAID/Sudan eventually transitions to a Country Development Cooperation Strategy (CDCS), the use of a methodology to improve development efforts through Page25 increased coordination and collaboration, testing of promising new approaches, and adaptation of approaches when appropriate to improve effectiveness, will be increasingly important focal points. To that extent, the evaluation will be expected to analyze each FOG recipient’s strategy and approach and make recommendations based on those findings. III. EVALUATION OBJECTIVES: The Automated Directive System (ADS) 203.3.6.1 requires that an evaluation is conducted when there is a distinct and clear management need to address an issue. This review is to critically examine the overall performance of the selected FOGs, as well as to analyze the efficacy of FOGs as means of civil society development programming in Sudan. The evaluation methodology and process shall address the questions outlined below for each FOG: 1. To what extent did each FOG meet its overall goal? The Contractor shall review the project performance of each FOG and establish the extent to which the intended goal and results have been met. What were the success factors and challenges that have hindered the achievement of the results? Analysis of performance shall also establish the degree to which the program design and management were conducive to post-project sustainability of the grantees and replication by others. 2. How effective were the FOGs as a mechanism for DG programming? The Contractor shall analyze how well the fixed obligation grants worked as a programming mechanism for DG activities in Sudan, especially at the particular time immediately after the separation of South Sudan and the early transition to a new country and Mission context. Was the FOG mechanism effective in meeting short-term results of the DG team’s strategy, or would another mechanism have been more appropriate? What were the pros and cons of using the FOG mechanism for these DG activities at the time they were used? 3. How realistic and appropriate was the design of the project? The Contractor shall review the project concept and design to assess whether it was responsive to the need at that time, realistic in approach, and appropriate in light of recipient capacity. To what extent did the design of each FOG, especially the way project milestones were identified and sequenced, influence the outcomes? 4. Were the appropriate stakeholders involved in the program and to what extent did the program promote better coordination and collaboration between them? The Contracto shall establish and assess the extent to which the critical stakeholders and inter-relationships were correctly identified, structured and involved in the delivery and management of the activity. To what extent did each project address the core issues for which its grant was given? IV. EVALUATION METHODOLOGY The Contractor shall propose a clear methodology to answer all the evaluation questions, using both quantitative and qualitative methodologies such as focus groups, structured interviews and/or questionnaires, as appropriate. The final evaluation report shall describe the techniques used in the methodology, including sampling techniques, and shall also include a brief assessment of the relative Page26 strengths and weaknesses of each method employed (e.g. possibly sampling bias, inadequate sample size, etc.). Preliminary analysis and review of relevant documentation will be conducted to gain an understanding of contextual issues, including transition priorities, legal authority and the cooperation framework. The Contractor shall also review project documents including grant agreements and reports. In analyzing program performance, the Contractor shall gather views and inputs of critical stakeholders who have been involved with each of the FOGs. These stakeholders include, but are not limited to, the USAID DG team and supporting offices, grantee personnel, associated civil society organizations, participants and beneficiaries and other donors in the sector. With regard to data quality, the Contractor shall be familiar with and follow USAID data quality standards for objectivity, validity, reliability, precision, utility and integrity and be able to apply them in the final report, by identifying such data limitations as may exist with respect to these standards (ADS 578.3.4.2 - http://transition.usaid.gov/policy/ads/500/578.pdf ) and ADS 203.3.5.1 - http://transition.usaid.gov/policy/ads/200/203.pdf ). Findings of the evaluation will be shared with USAID and other development parties to inform of better administration and execution of democracy and governance grants in Sudan and other post-conflict developing countries. V. PROGRAM INFORMATION The following information documents and sources are available and relevant to the review: USAID policy documents (FOGs, USAID Forward, etc.) Original Unsolicited Proposals for each FOG. Grant documents and work plans. Grant deliverables associated with agreed milestones. Final reports. VI. EVALUATION TEAM COMPOSITION The Contractor shall form an evaluation team consists of three experts, one team leader and two team members who should possess the skills and experiences below: Team Leader: Page27 Master’s degree or higher in international studies, sociology, political science, anthropology, or other social science discipline; or demonstrated knowledge and experience in monitoring and evaluation systems. Fluency in English. Fluency or proficiency in Arabic is strongly desired. Qualifications and demonstrated experience in the design and management of evaluations. Five to ten years of experience with democracy, governance, civil society, and gender issues. Extensive experience in Sudan, Africa, and other Arab countries will also be considered. Demonstrated strong research, analytical, and writing skills. USAID programming experience is desirable. Two Team Members: Bachelor’s degree or higher in international studies, sociology, political science, anthropology, or other social science discipline. Ability to communicate with Team Leader in either English or Arabic is required. Native Arabic speaker with fluency in English. Demonstrated past experience in the design and management of evaluations is desired. At least three years of experience with democracy, governance, civil society, and gender issues preferable. Must be Sudanese citizen. Strong research, analytical, and writing skills is desirable. USAID programming experience is desirable. VII. DELIVERABLES The Contractor shall deliver the following outputs to USAID/Sudan: Deliverable Level of Effort Total Level of Effort 1. In-brief meeting for an introduction of the evaluation team, discussion of the SOW and initial presentation of the proposed evaluation work plan. 1 day x 3 persons 3 Page28 2. An implementation report submitted to USAID within one week after the in-brief. The report will include: activity to be undertaken, including field work. -grantees and field sites to be visited, data collection instruments. 2 days x 3 persons 6 3. Field work/Data collection 7 days x 3 persons 21 4. Oral debriefing to USAID, FOG recipients, and selected partners to present key findings prior to submission of draft report. 1 day x 3 persons 3 5. Draft evaluation report in both hard copies (2) and one electronic copy for review by USAID. *Please see the Illustrative Report Outline at the end. Requirements for the evaluation report are also attached. 3 days x 3 persons 9 6. Meeting between Evaluation Team, USAID, and possibly FOG recipients to discuss draft evaluation report questions, comments, and desired edits. (Attendees will be agreed upon with USAID.) 1 day X 3 persons 3 7. Final evaluation report in both hard copies (2) and one electronic copy incorporating feedback from USAID. 1 day x 1 person 1 Total 46 VIII. ROLES & RESPONSIBILITIES Contractor: Conduct evaluation, per the requirements set forth in this scope of work. Make own meeting and logistical arrangements. USAID: Appoint a point of contact for the assignment to coordinate USAID inputs. Approve the Contractor’s methodologies and work plan. Facilitate initial introduction of the Evaluation Team to grantees. Participate in briefings. Review draft evaluation reports and provide feedback. Page29 Sign off on final report. IX. ILLUSTRATIVE REPORT OUTLINE Introductory Section of Report: Cover page: Title of the study, the date of the study, recipient’s name, name of the Evaluator. Preface or Acknowledgements (Optional) Table of Contents List of Acronyms Executive Summary: Stand-alone, 1-2 pages, summary of report. This section should not contain any material not found in the main part of the report. Main Part of the Report: (Not to exceed 15 pages) 1. Introduction/Background and Purpose: Overview of the final evaluation. Covers the purpose and intended audiences for the final evaluation and the key questions as identified in the SOW 2. Study Approach and Methodology: Outline how the evaluation was designed, conducted, and what specific methods were used. 3. Findings: This section, organized in whatever way the team wishes, must present the basic answers to the key evaluation questions, i.e., the empirical facts and other types of evidence the study team collected, including any assumptions. 4. Conclusions: This section should present the team‘s interpretations or judgments about its findings. 5. Lessons Learned: In this section, the team should present any information that would be useful to people who are designing/manning similar or related new or on-going programs in Sudan 6. Recommendations: This section should make it clear what actions should be taken as a result of the study. X. CRITERIA TO CHECK THE QUALITY OF THE EVALUATION REPORT The evaluation report shall represent a thoughtful, well-researched and well-organized effort to objectively evaluate what worked in each project, what did not work, and why. The evaluation report shall address all evaluation questions included in the scope of work. The evaluation report shall include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by USAID/Sudan. Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides will be included in an annex in the final report. Page30 Where applicable, evaluation findings shall take into account each project’s impact on both males and females. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). Evaluation findings shall be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people‘s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence. Sources of information need to be properly identified and listed in an annex. Recommendations need to be supported by a specific set of findings. Recommendations shall be action-oriented, practical and specific, with defined responsibility for the action. Page31 Annex (2) Ref: Purchase Order AID-66-O-12-00007 with Partners in Development Services (PDS) Implementation Plan FOGs Evaluation Following being introduced by USAID to grantees the evaluation team will contact grantees to schedule day and time e for each interview (within the specified time frame set in the plan. It is not practical for them to schedule dates for field work before receiving the 1st installment. However, it is likely that the team can only interview USAID partners in Khartoum before the Eid which is due in less than two weeks time. Therefore, the field work can be conducted immediately after the Eid. USAID-DG CSO/Entity People to Meet Methods to Use Time-Frame IDSC and SUGDE COR/Project Management Specialist Acquisition & Assurance Specialist Any other available staff USAID-DG invites to attend. Structured Interview 16-22 October 2012 1. SICM2 CSO/Entity People to Meet Methods to Use Responsibility Time-Frame IDSC and SUGDE Director IDSC & Chair SUGDE Structured Interview Gadallah & Omer 16-22 October 2012 SICM Coordination Committee (SCC) Members to SCC Focused Group Discussion Gadallah & Omer 16-21 Octobrt 2012 SICM Secretariat Training and media coordinators Structured Interview Gadallah & Omer 16-21 October 2012 Beneficiaries (Trained trainers and Focal Points) Focal point and trainer in five selected states Sampling techniques Structured Interview To be set after receiving 1st installment 2. NCF Board/Executive Committee Chairperson & Secretary General Structured Interview Gadallah & Omer 16-21 October 2012 Trainers Structured Gadallah & Omer 16-21 October 2012 Beneficiaries Sampling techniques Structured Interview Gadallah & Omer To be set after receiving 1st installment Page32 3. AUW AUW International and External Relations office Rural, Extension, Education & Development (REED)- project’s coordinator Vice Chancellor Director of International and External Relations office – Project Coordinator Dean, REED Trained core group Structured interview Structured Interview Structured interview Nabawia Nabawia Nabawia 16-21 October 2012 Dalanj Peace Studies Centre Director Involved Staff MA gradutes Focus Group Discussion & semi￾structured interview Nabawia To be set after receiving 1st installment Gedarif Peace Studies Centre Director Involved Staff MA gradutes Structured Interview Nabawia To be set after receiving 1st installment Gedarif Faculty of Community Development Dean Involved Staff Msc gradutes Focus Group Discussion & semi￾structured interview Nabawia To be set after receiving 1st installment SCLS SCLS Governing Body SCLS leadership and office staff Structured Interview Gadallah & Omer 16-21 October 2012 AUW AUW-RIG/DPR Director of RIG/DPR Project’s Coordinator Database & webpage staff Structured Interview Gadallah & Omer 16-21 October 2012 Debriefing to USAID before submission of draft report Oral debriefing/Discussion Nabawia, Gadallah & Omer Towards the end of the first week of November3 Submission of draft report Second week of November Meeting between USAID, Evaluation team and possibly FOG recipients Second week of November Submission of Second week Page33 final evaluation of November Annex (3) Ref: Purchase Order AID-66-O-12-00007 with Partners in Development Services (PDS) FOGs Evaluation Check List 1. USAID  How effective were the FGOs as a mechanism for DG programming compared to previous programming mechanisms?  How would you assess FOG mechanism in terms of paperwork, time involved when reviewing project proposals and awarding grants?  What are the risks of focusing on milestones/deliverables rather than on usual procedures followed by other donors? And if such risk exists how you would manage reimbursing already awarded funds?  How would assess access of Sudanese CSOs to FOGs?  Did you have any incident where the situation requires introducing changes in the milestones or the program objectives in consultation with USAID?  Were the appropriate stakeholders involved in the program?  To what extent did the program promote better coordination and collaboration between stakeholders?  What are the criteria for selecting projects?  To what extent did the project meet its rationale and overall objectives?  What were the opportunities missed or accomplishments that fell short of potential or expectations? 2. Grantees  Director IDSC & Chair SUGDE  How effective were the FGOs as a mechanism for DG programming?  How would you compare FOGs Application and awarding procedures in terms of paperwork, time involved and scope of funding to other donors such as UNDP and USIP?  Cons and Pros of FOGs  How realistic, appropriate and flexible was the design of the project?  To what extent did the project meet its overall goal?  SICM-CC  To what extent did the project meet its overall goal?  What are the Challenges and constraints met?  What elements of sustainability are incorporated in the project design? Pending receipt of first installment  How did FOG funding help in achieving SICM objectives  What are the key successes?  What are the lessons learned?  AUW INVOLVED STAFF:  Levels students and institution.  What are the key successes?  What are the lessons learned?  What are the missed opportunities? Page34  The quality of the Reader; incorporation of human rights, Democracy and Governance, user friendly as it will be used as trainer guideline and the validity of reader’s component o be part of University Rural Extension Programme;  Validity of information to be disseminated during the forthcoming fieldtrips.  The quality of brochures; inclusiveness of the main messages/information about human rights, democracy and governance  M.Sc. Students  Relevance of the programme to their communities’ needs and expectations.  Efficiency of MSc graduates in disseminating the gained knowledge.  The impact of the programme at graduates and university level.  Sustainability of impact.  Recommendations for further assistance.  Relevance of the programme to their communities’ needs and expectations.  Means to integrate the gained knowledge in the messages of rural Extension Field trips.  Contribution in building the capacities of home university colleagues and others works in the field of relevance.  Impact of the programme at personal and institutional level.  RIG/PDR-AUW  Achievements so far and the need for possible adjustment.  Relevance of the activities being undertaken.  Challenges and constraints encountered  Elements of sustainability are incorporated in the project design?  How did FOG funding help in are on track.  achieving RIG/DPR objectives;  What are the key successes?  What are the lessons learned so far?  What are the missed opportunities?  Impact of the project at network and institutional level.  Recommendations with regard to the remaining project period and beyond. Page35 (Annex (4) Ref: Purchase Order AID-66-O-12-00007 with Partners in Development Services (PDS) FOGs Evaluation Annex (4) – People Met SICM Entity People Met Methods to Use Responsibility Time-Frame IDSC and SUGDE Dr. Girshab (Director, IDSC & Chair SUGDE) Ms Manal Allah Gabo Semi-structured Interview Gadallah & Omer 17 October 2012 SICM States and Training Coordinator Awad Al Kareem Semi-structured Interview Gadallah & Omer 17 October 2012 SICM Secretariat Mohammed Fielabi, Media coordinators Semi-structured Interview Gadallah & Omer 17October 2012 SICM Coordination Committee (SCC) Dr Girshab Omiema Al Mardi Mashayir Mohamed Al Amin Awad Al Karim Khalid Laila Ali Al Haj Focused Group Discussion Gadallah & Omer 18 October 2012 SICM Focal Points Port Sudan4 Aisha Hamad & Al Tayib Al Hadi (East Sudan Hawa Development Society) Semi-structured Interview Omer 26 October River Nile5 Ahmed Fuaad Semi-structured Interview Omer 27 October Kassala Al Tahir Bashari (AL Sharg Center for Culture and Legal Aid) Jawahir Awad Allah (Women Organization for Peace & Development) Lutfi Fouad Abdo Semi-structured Interview Semi-structured Interview Phone Interview Amin, Ilham Ilham Ilham 30 October 30 October 31October 4 Husna Tahir”Hai Al Shajarah Society” was not reachable by phone during the evaluation visit to Port Sudan 5 The other 2 focal points reside in Shendi and Berber Page36 (Afaq) Gedarif Friends of Peace and Development organization. Haram Sir Elkhatim Haram Zienab Women Development Organization Fatma Samahen , Mohmed Omer Hajana Raidat AZA: Faiza Wargo Ridaat 1st November 2012 South Kordofan 6 Yousif Marfaeen, Director Elgoghan Organization for Development and Reconstruction Kamilia koko - Director, Nuba Women for Education and Development (NUWEDA) Blue Nile 7 Al Rasheed Abdel Majid; Paralegal Association Wafaa Sirel Khatim, Friends of Peace & Development Society Gazira Bakikir Bello and Afraa Ibrahim, the Sudanese Center for Democracy & Development 6 All met in Khartoum 7 First met in Khartoum whereas second met in Gedarif Page37 South Darfur Mohammed Zakaria, Union of Plastic Artists 31 October 2012 Geda Wafaa Sirel Khatim of the Friends of Peace and Development Society AUW Prof. Balghis Badri - Director of the Regional Institute of Gender, Peace, Diversity and Rights. Dr. Nafisa Mohamed Badr i - AUW - Director, International & External Relations. Dr. Abu Bakr Abdelazeem -AUW - Dean of school of Rural Extension, Education and Development. Uz. Hafiz Mohamed Malik - Assistant Coordinator - Constitution Building Project - Training Component. Uz. Sunia Aziz Malik - Assistant Coordinator - Constitution Building Project – Legal Analysis Component. Uz. Abu Bakr Ahmed Mohamed Elkhair - Post - Graduate Registrar - Regional Institute of Gender, Peace, Diversity and Rights. Page38 Uz. Rasha Awad Abdella - Assistant Coordinator - Constitution Building Project – Directory and Data Base Component. Alia Abdella Ali Elfaig - MSc Student - Gender and Migration. Ikhlas Omer Ibrahim Kajo - MSc Student - Gender and Migration. Nada Ibrahim Taha – MSc Student￾Gender and Migration. Hamida Ali Aldow – MSc Student￾Gender and Migration. Huda Ibrahim Aglian – MSc Student - Gender and Migration. Thoiba Hashim Galad – MSc Student - Gender and Migration. Aida Hakim Abashir – MSc Student - Gender and Migration. Amna Ahmed Mukhtar – MSc Student - Gender and Migration. Ezdihar Omer Mohamed – MSc Student - Sustainable Development Shahinda Saeed Ali - MSc Student￾Sustainable Development Page39 Civic Forum Gadarif . Disabled Union Husham Mai A. Gadir 1st November 2012 Kassala Mohmoud A Tirik, M A Tirik & Ahmed Tahir (Delta Semi-structured Interview Gadallah/Ilham 25 October 2012 Obied Voluntary Society Yousif Abdallah Mahmoud Sami Mohammed Al Nour 31 October 2012 SCLS Ali Sulieman Nawal Abu Gisie3sa Naafisa Badri Page40 (Annex 5) Ref: Purchase Order AID-66-O-12-00007 with Partners in Development Services (PDS) Evaluation of USAID Fixed Obligation Grants (FOGs) Debriefing Meeting Thursday 29th November, 2012 From 11:00 to 13:00 PDS office Attendance from – USAID: 1. Zaria Elnour Ali Sulieman Attendance from – FOGs : 2. Prof. Ali Sulieman - SSCL 3. Dr. Hassan Abdel Ati - NCF￾edgesudan@yahoo.com 4. Mr. Hafiz A. Elbasheer - AUW-RIG/DPR- balghis.badri@yahoo.com 5. Mr. Awad Elkareem Fadl Almola-SICM – hsmba32@gmail.com Attendance from – PDS: 6. Dr. Ahmed Abusin 7. Mr. Omer Sir Elkhatim 8. Mr. Gadallah Elradi 9. Mrs. Nabawia Harbi Agenda of meeting: Debriefing on the evaluation’s key findings, lessons learned and recommendations. Page41 Annex (6) Partners in Development Services (PDS) Vision and Mission ‘Partners in Development Services’ is a Sudanese consultancy group founded to mobilize national expertise in a new multi-disciplinary approach with a view of developing new and appropriate approaches and models for development. The challenges for development of the forthcoming millennium call for new thinking and strategies that cope with the tremendous advances in knowledge that surely completely globalizes through speedy information super-highways. Partners is unique among sister service organizations, in its unfailing belief in shaping the mainstream development efforts through participation of people as the main responsible about their development. Our role therefore, besides availing the necessary technical assistance, rests on facilitation of the development process as a whole and help influence related policy and practice changes. Taking the special Sudanese context, where partners mainly operate, social, agricultural and community development themes make the overriding challenge domain. Issues of conflicts, peace culture, are not under estimated within this context. Moreover, Partners competence and ability to mobilize the necessary expertise and resource persons defined our course of action to meet that challenge. What we can provide therefore, are technical assistance, research and studies, and training inputs. Mainstreaming of gender, environment and capacity building of grassroots organizations and support to civil society institutions is what makes Partners a development need in Sudan. PDS is a Sudanese Consultancy Firm with a main focus on research and training (Annex 6). During 2012 PDS was assigned by USAID/OFDA, AECOM, UNDP, JICA and the World bank to assess and evaluate some of the projects for which they had provided funds.