1 A Performance Evaluation of the Effectiveness and Sustainability of USAID Kenya and East Africa- Supported Activities at the Intergovernmental Authority on Development EVALUATION REPORT JUNE 3RD, 2016 This evaluation report was prepared independently and produced for review by the U.S. Agency for International Development (USAID). It was prepared by Mr. Colin Reynolds and Mr. Manasseh Wepundi for the Evaluation Services and Program Support team of International Business & Technical Consultants, Inc. USAID/Kenya and East Africa Evaluation Services and Program Support A Performance Evaluation of the Effectiveness and Sustainability of USAID (Kenya and East Africa) Supported Activities at the Intergovernmental Authority on Development June 3rd, 2016 Prepared for Ms. Jaynnie Mulle United States Agency for International Development/Kenya c/o Embassy of the United States United Nations Avenue, Gigiri P.O. Box 629, Village Market 00621 Nairobi, Kenya Prepared by International Business & Technical Consultants, Inc. 8618 Westwood Center Drive Suite 400 Vienna, VA 22182 USA DISCLAIMER The authors’ views expressed in this report do not necessarily reflect the views of the U.S. Agency for International Development (USAID) or the United States Government. Acknowledgements The evaluation team wishes to thank a wide range of people for their support. Intergovernmental Authority on Development (IGAD) Secretariat staff, particularly the Executive Secretary, H.E. Ambassador (Eng.) Mahboub Maalim, made themselves available to the team and facilitated many meetings with other IGAD agencies. Many U.S. Agency for International Development (USAID) Kenya/East Africa staff provided documents and direction through all stages of the evaluation. Of special note, Ms. Jaynnie Mulle was invaluable to the team from the start. She attended the team planning meeting and traveled to Djibouti to facilitate meetings for the team and with respondents throughout the region. She ensured that the team had what they needed when they needed it; her responsiveness was key to the team’s success. The team also expresses their gratitude to all interviewees and agency respondents for their time and valuable insights. Finally, the team would like to acknowledge the considerable support provided by the International Business and Technical Consultants, Inc. (IBTCI) Evaluation Services and Program Support (ESPS) Office in Kenya, without which this evaluation would not have been possible. Table of Contents Acknowledgements .......................................................................................................................... iii Table of Contents ............................................................................................................................. iv Glossary of Terms .............................................................................................................................. i Acronyms and Abbreviations ........................................................................................................... ii Executive Summary ......................................................................................................................... iii 1. Introduction ............................................................................................................................... 1 1.1 Intergovernmental Authority on Development (IGAD)...................................................................... 1 1.2 IGAD Conflict Early Warning and Response Mechanism ................................................................... 2 1.3 IGAD Drought Disaster Resilience Sustainability Initiative (IDDRSI) .................................................. 3 1.4 IGAD Climate Prediction and Application Center .............................................................................. 3 2. Evaluation Objectives and Questions....................................................................................... 4 2.1 Evaluation Objectives ........................................................................................................................ 4 2.2 Evaluation Questions......................................................................................................................... 4 3 Methodology and Limitations ................................................................................................... 5 3.1 Approach to Field Work ..................................................................................................................... 5 3.2 Site Selection ..................................................................................................................................... 5 3.3 Target Groups and Selection Methods .............................................................................................. 5 3.4 Data Collection Methods ................................................................................................................... 6 3.4.1 Document Review ................................................................................................................................................. 6 3.4.2 Formal Information Requests ............................................................................................................................. 6 3.4.3 Key Informant Interviews .................................................................................................................................... 6 3.5 Data Management ............................................................................................................................ 6 3.6 Ethical Considerations ....................................................................................................................... 7 3.7 Limitations ......................................................................................................................................... 7 3.8 Analysis ............................................................................................................................................. 7 4 Findings, Conclusions, and Recommendations ....................................................................... 8 4.1 Introduction ....................................................................................................................................... 8 4.2 IGAD Accomplishments ..................................................................................................................... 8 4.2.1 Evaluation Question 1 ................................................................................................................................... 10 4.2.2 Evaluation Question 2 ................................................................................................................................... 14 4.2.3 Evaluation Question 3 ................................................................................................................................... 16 4.2.4 Evaluation Question 4 ................................................................................................................................... 19 4.3 CEWARN ..........................................................................................................................................20 4.3.1 CEWARN Evaluation Question 1............................................................................................................ 20 4.3.2 CEWARN Evaluation Question 2 ................................................................................................................... 27 4.3.3 CEWARN Evaluation Question 3 ................................................................................................................... 30 4.4 IDDRSI ..............................................................................................................................................31 4.4.1 IDDRSI Evaluation Question 1 .................................................................................................................. 33 4.4.2 IDDRSI Evaluation Question 2 .................................................................................................................... 34 4.4.3 IDDRSI Evaluation Question 3 ......................................................................................................................... 37 4.5 ICPAC ...............................................................................................................................................38 4.5.1 ICPAC Evaluation Question 1 ..................................................................................................................... 39 4.5.2 ICPAC Evaluation Question 2 ..................................................................................................................... 41 4.5.3 ICPAC Evaluation Question 3 ..................................................................................................................... 42 5 Overall/Cross-cutting Recommendations .................................................................................. 43 5 Summary Conclusion .............................................................................................................. 44 ANNEXES ........................................................................................................................................ 46 Annex 1: Map of Participating Countries .....................................................................................................46 Annex 2: Statement of Work .......................................................................................................................47 Annex 3: Evaluation Team Member CVs ......................................................................................................55 Annex 4: Matrix of Questions and Sub-Questions .......................................................................................64 Annex 5: Complete List of Documents Reviewed .........................................................................................68 Annex 6: Responses to Information Requests ..............................................................................................72 Annex 7: Instruments....................................................................................................................................73 Annex 8: List of Respondents for the KIIs .....................................................................................................89 Annex 9: Numerical Analysis of KII Data ......................................................................................................92 Annex 10: Stakeholder Misconceptions ........................................................................................................95 Annex 11: Lessons Learned ...........................................................................................................................98 Annex 12: Recommendations for Revised CEWARN System ......................................................................109 Annex 13: Responses to IGAD (General) and CEWARN Secretariat Comments on Draft Evaluation Report 114 Annex 14: Evaluation Comments on Revised CEWARN System Update ....................................................126 Annex 15: Additional Comments on IGAD Evaluation Report from the CEWERUs .....................................157 i Glossary of Terms Client capture: When a specific client of a service provider exerts sufficient influence over the provider to influence their outputs in ways that may be detrimental to other clients Core business planning: A specific form of focused planning that includes the following steps:  Clear identification of an agency’s priority clients in the region;  Clear identification of the products and services that are most needed (and, therefore, would be most highly valued) by high-priority clients;  Clear identification of which of these key products and services are satisfactory and already available from others in the region;  Clear identification of the remaining key products and services within an agency’s capacity;  Clear identification of real-world constraints that may limit an agency’s mandated roles, powers, or aspirations; and  Specifying and monitoring of robust performance and outcome indicators, some of which specifically measure coordination between agencies Domestication: The national adoption of all roles related to implementation of an externally￾provided regional system to potentially include roles initially intended for the external regional organization that provided the system Empire building: When an agency attempts to raise its own profile at the expense of its functional associations with its parent organization External donor: In the contexts of key informant interviews, refers to development partners other than the U.S. Agency for International Development (USAID) IGAD member states: Djibouti, Ethiopia, Kenya, Somalia, Sudan, South Sudan, Uganda and Eritrea Information requests: Requests for information sent in writing by the evaluation team at the outset of the exercise in response to gaps noted in existing reading material Mandate: The definition of the extent and limitations of an agency’s powers to act, usually found in the authorizing document that creates the organization Member State agencies: Governmental agencies and/or offices, such as the country early warning and early response units (CEWERUs) or national meteorological agencies Multilateral agency: An intergovernmental organization comprising more than two countries On paper: The perceived operating environment of an activity or organization based solely on assumptions made during a planning process Process-oriented approach: An approach to planning that focuses more on what is done than what is achieved Rapid response fund: A kitty set aside for supporting urgent conflict prevention or mitigation activities within the Conflict Early Warning and Response Mechanism (CEWARN) Real world: The operating environment of an activity or organization inclusive of all practical and current constraints, including those that contradict assumptions made during a planning process Sessional paper: Any document tabled in the parliament or filed with the clerk of parliament Situation Room: A designated office or room in a CEWARN or CEWERU where the latest conflict early warning data is processed and responses are developed Strategy: An overarching planning document that is intended to direct annual work planning Tragedy of the commons: When some consumers of a common resource take advantage of their access to that resource at the expense of other consumers Face value: Taken as written, rather than trying to interpret questions on a broader basis ii Acronyms and Abbreviations ACT Act Change Transform! (Formerly PACT) CCC CEWARN Country Coordinator CEWARN Conflict Early Warning and Response Mechanism CEWERU Conflict Early Warning and Response Mechanism Unit CGIAR Consultative Group for International Agricultural Research CIDA Canadian International Development Agency COF climate outlook forum COMESA Common Market for Eastern and Southern Africa CPP country program paper CPS Committee of Permanent Secretaries CSO civil society organization DfID Department of International Development DMCH Drought Monitoring Center, Harare DMCN Drought Monitoring Center, Nairobi EAC East African Community ESPS Evaluation Services and Program Support EU European Union FAU Financial Administration Unit FM field monitor GIZ Gesellschaft für Internationale Zusammenarbeit, the German international development agency IBTCI International Business & Technical Consultants, Inc. ICPAC IGAD, Climate Predication and Application Center IDDRSI Drought Disaster Resilience and Sustainability Initiative IGAD Intergovernmental Authority on Development IGADD Intergovernmental Authority on Drought and Development ILRI International Livestock and Research Institute ISAP institutional strengthening action plan ISC IGAD Steering Committee KII key informant Interview KM knowledge management LSGA Limited Scope Grant Agreement M&E monitoring and evaluation NGO Nongovernmental organization NMHS National Meteorological and Hydrological Services NRI national research institutes PCU Platform Coordinating Unit PDI Office of Program Development and Implementation PREPARED Planning for Resilience in East Africa through Policy, Adaptation, Research and Economic Development PSC Program Steering Committee RRF Rapid Response Fund SWOT Strength, Weakness, Opportunities and Threat TCEW Technical Committee for Early Warning UN United Nations UNDP United Nations Development Programme USAID U.S. Agency for International Development USAID/EA U.S. Agency for International Development/East Africa iii Executive Summary International Business & Technical Consultants, Inc. (IBTCI) was awarded the Evaluation Services and Program Support (ESPS) contract on August 28, 2013. This indefinite delivery/indefinite quantity contract provides monitoring and evaluation (M&E) services to the U.S. Agency for International Development (USAID) Kenya Office of Population and Health and to USAID’s East Africa (EA) Regional Health and HIV/AIDS and Program Development and Implementation offices. The contract is implemented by discrete task orders. The ESPS Task Order Response Team received a scope of work on October 28, 2014 to conduct an evaluation on the effectiveness and sustainability of USAID/EA-supported activities at the Intergovernmental Authority on Development (IGAD). Since 2006, the USAID/EA Mission has provided direct financial support to three IGAD programs through a Limited Scope Grant Agreement. This current performance evaluation focuses on three USAID/EA-supported activities: the Conflict Early Warning and Response Mechanism (CEWARN); the IGAD Drought Disaster Resilience and Sustainability Initiative (IDDRSI) implemented through the Platform Coordinating Unit (PCU); and the IGAD Climate Prediction and Application Center (ICPAC) under the Planning for Resilience in East Africa project, which is part of the Policy, Adaptation, Research and Economic Development (PREPARED) program. ESPS received a task order from USAID on July 2, 2015, to evaluate the effectiveness and sustainability of USAID/EA-supported IGAD activities. Prior to receipt of the task order, the ESPS team worked with USAID to design a practical methodology that would enable the evaluation team to answer the evaluation questions, while balancing cost and time. The evaluation questions, which are listed below, were extensive and often multi-layered. General Evaluation Questions 1. To what extent do the various IGAD efforts supported by USAID coordinate to achieve IGAD’s overall objectives? How could these efforts better complement each other to achieve IGAD’s overall objectives? 2. To what extent are current donor structures/relationships conducive to IGAD achieving its organizational objectives and how can these be improved? What are the primary areas USAID should focus on to effectively advance IGAD’s agenda? 3. To what extent are the CEWARN, IDDRSI and ICPAC activities sustainable? What is the extent of IGAD member states’ financial, political, and in-kind support for these activities and how can that support be strengthened? 4. What has been IGAD’s experience with its USAID partnership? Are there areas that can be improved and if so, which areas are those? Program Specific Evaluation Questions CEWARN 1. To what extent is CEWARN on track to fully implement its strategic framework? What has worked well and what has not? Are all CEWERUs equally effective in advancing the strategic framework or are there varying levels of success? What factors contribute to any differing levels of success and what lessons can CEWERUs learn from each other? 2. Are CEWARN and its national level CEWERU interventions meaningful and responsive to cross-border conflicts that occur in the region? If so, what factors have made that possible? What have been the main challenges for CEWARN and the CEWARUs in addressing cross-border conflicts? To what extent can CEWARN be expected to effectively contribute to advancing peace and stability in the East Africa region it serves? iv 3. To what extent do civil society organizations in the East Africa region work with and support CEWARN and/or the CEWERUs? What recommendations do the regional stakeholders, including CSOs, have for improving the effectiveness of the CEWARN and CEWARU programs? IDDRSI 1. How has IGAD leveraged USAID support to strengthen the PCU to implement the IDDRSI strategy? 2. Since its inception, how has USAID support developed the foundations for knowledge management in IGAD? What challenges have been experienced by IGAD in operationalizing the knowledge management system? How have these challenges been handled? 3. USAID/EA has supported the institutional capacity strengthening of IGAD by providing resources for IGAD to hire core finance and administration staff. To what extent have the expected results and/or outcomes been achieved? What steps is IGAD taking to assume responsibility for funding of these staff to avoid dependence on USAID? What is the anticipated timeline for IGAD to fully assume funding responsibility for these staff? ICPAC 1. To what extent has ICPAC been effective and efficient in achieving its strategic objectives and responding to climate change priorities? 2. To what extent are USAID-supported programs responding to thematic priorities as outlined in ICPAC’s strategic plans and other regional strategies agreed to by member states? 3. How are the current systems and processes of ICPAC helping or hindering it from meeting regional climate change demands and priorities? In both the in-brief and midterm update, the team noted that it would be difficult to respond to each question with a similar level of detail or evidence and that there was a risk of significant overlap with the recently-conducted institutional risk assessment of IGAD (also funded by USAID). To retain a range of perspectives, the direction provided to the team by USAID was to retain the extensive scope, but carefully caveat data limitation. This external performance evaluation will help determine which USAID-funded programs and activities of IGAD, CEWARN, IDDRSI, and ICPAC are working well and why, and which have not achieved expected results and why. It will also provide USAID/EA with information to guide modifications and mid-course corrections, if necessary, and help IGAD improve the performance and results of its CEWARN, IDDRSI, and ICPAC programs. Intended users include: the IGAD Secretariat and IGAD Institutions (CEWARN, the IDDRSI Platform, and ICPAC); the IGAD Institutional Strengthening Action Plan (ISAP) Coordination Group; IGAD member states; USAID/ EA/Kenya; USAID/Washington (particularly the Resilience Secretariat) and USAID/EA bilateral missions (especially Ethiopia and Somalia). ESPS established a two person team that included a team leader and a senior investigator to conduct the evaluation. This team collected data from 106 stakeholders (47 females, 59 males) within the various institutions and organizations working with IGAD in four capitols in the region, Djibouti (Djibouti), Addis Ababa (Ethiopia), Nairobi (Kenya), Kampala (Uganda), and Juba (South Sudan). Interview methodologies included qualitative interviews, key informant interviews (KIIs), and group interviews. The evaluation team used a mixed methods approach to data gathering, including: 1) document review—primarily progress reports and statistical information available from IGAD and from other relevant reports and findings; 2) quantitative data received directly v from relevant organizations through formal (written) information requests defining the specific types of data sought; and 3) qualitative data gathered through KIIs. (See Annex 4: Matrix of Questions and Sub-Questions). IGAD and its agencies face many small challenges that are mostly symptoms of one overarching issue—the unpredictability of IGAD’s funding from member states. Shortfalls or delays in member states’ contributions have had marked effects on all IGAD activities. For example, sometimes IGAD agencies receive their internal budgets near the end of the annual implementation cycle, which makes it difficult for the agencies to function effectively. IGAD faces a challenging situation regarding external donors. Though it may be preferable that IGAD only accepts external funding that aligns with its existing work plans, their work plans are often not specific enough to prevent donor preferences substantially affecting immediate decision making. This is especially true given that IGAD is almost entirely reliant on external funding for operational activities. This situation impedes medium-term planning, makes design of outcome level M&E difficult, and continually stretches the limited permanent capacity of the organization and its agencies. Donors view these performance issues with growing frustration, noting IGAD’s inability to conduct focused work planning, its failure to provide clear outcome reporting, and its constant administrative delays (due to under staffing). Donors and IGAD share responsibility for the current performance challenges. However, all donors consulted expressed frustration at IGAD’s ability to demonstrate clear outcomes of its initiatives. IGAD will need to decisively and immediately address this issue. Without change, IGAD risks losing funding and external donors could lose a unique vehicle for providing effective regional assistance. The IGAD Secretariat has delivered valuable, high-level products. It mediated the South Sudan internal conflict and facilitated the country’s economic infrastructure projects. Regardless of current performance, there remains a shared perception among many stakeholders, including donors, national agencies, nongovernmental organizations (NGOs) and others that IGAD’s platforms and agencies (currently funded by USAID) continue to possess the ability and potential to deliver unique and important regional products and services. IGAD appreciates USAID’s leverage in the donor community and considers its USAID partnership one of strategic importance. This viewpoint hinges on IGAD’s expectation that USAID will use its influence to assist in coordination of donor support for IGAD in line with the Paris Declaration on Aid Effectiveness. USAID’s influence could be particularly helpful given the challenges IGAD faces in managing multiple donor reporting frameworks, multiple institutional assessments, and the time implications of responding to individual donor demands. Concerns exist, however, about USAID’s disbursement of funds to IGAD and its agencies. Of USAID’s $1.6 million awarded to the CEWARN Secretariat, for example, the IGAD agency has only received $500,000. While slow compliance with USAID accounting requirements was a factor in this case, the delay has created substantial cash flow challenges that have negatively affected implementation. The ICPAC team also has concerns regarding the change from quarterly to monthly fund disbursements, which means more time spent liquidating funds. As a key recommendation, USAID should continue to assist IGAD in its current areas of focus. However, funding should be contingent on the relevant IGAD agencies and platforms developing, with USAID assistance, work plans that are more strongly focused on delivery of specific outcomes to address and resolve the current performance challenges. Regarding the specific program components of IGAD covered in this evaluation: vi IDDRSI has had a slow start, but is gradually producing valuable coordination mechanisms. These include a regional programming paper, country programming papers and a Resilience Analysis Unit. Also, while IGAD has an important regional role in knowledge management, a lack of clear definition and delineation of this role could lead to duplicative and competing efforts between IDDRSI and other IGAD entities that collect or store data. USAID has supported administrative support positions and technical positions. In leveraging USAID assistance to strengthen the PCU to implement the IDDRSI strategy, IGAD has utilized all of its administrative support positions. Technical positions have been focused on specific IDDRSI projects contributing to the promotion of IDDRSI country programming papers across the board. Despite the urgency to make USAID-funded positions permanent, to date no steps have been taken to achieve this end. As such, it is not likely that the PCU will absorb these positions by the end of this funding cycle. ICPAC is already producing regional/seasonal forecasting products and services that are highly valued in the region, but the center has too few staff to cope with constant pressure from donors and others to increase its range of products and services beyond provision of regional and seasonal climate forecasting to member states, member state’s meteorological agencies, and other clients. Attendees express strong support and appreciation for the value of the climate outlook forums facilitated and conducted by ICPAC. These forums offer invaluable opportunities for joint technical discussions regarding the interpretation and implications of climate data at both the expert and practitioner levels. ICPAC has been successful in broadly communicating its forecasts and conveying implications of these forecasts. For example, the center has played a key role in communicating the likely effects of the current El-Niño to governments and other stakeholders. Governments have incorporated this information in public policies and action plans and considered it in their decision-making and resource allocation. The early warning and response mechanisms initiated by CEWARN have demonstrated strong success, especially in relation to addressing cross-border pastoralist conflicts. This success is more visibly felt in some countries and not in others due to the different levels of advancement by the CEWERUs. Recent attempts to change the CEWARN system may not have considered incorporating and consolidating past successes. While the new system does simplify both incident and situation reporting, it appears to complicate the administrative aspects of data handling and analyses (e.g., by moving from one to five National Research Institutes per member state) and appears to promote an academic or textbook approach rather than local functionality. It is also of concern that even the ‘simplification’ of raw data collection under the proposed new system remains reliant on responses to subjective ordinal (rating) scales, rather than on verifiable criteria. Potential for inter￾observer variation in responses to such subjective scales would in itself suggest they are a questionable input into regional conflict-related analysis algorithms. The recent stalling of the functionality of CEWARN’s early warning system is due to the sudden reorientation of the data collection approach. While this is partially due to a lack of funding for the system, it is also because the reorientation of the data collection approach may be practically infeasible. This infeasibility may be a result of a lack of support by many key stakeholders, including the civil society organizations (CSOs) that it purports to benefit. For four to five months in Ethiopia, and for close to a year in Kenya, South Sudan, and Uganda, field monitors had to halt data collection due to the termination of data collection contracts. While certain USAID staff have vii expressed support for this reorientation, the full basis on which the decision to implement this change was made remains unclear to the evaluation team. Cross-border conflicts pose another challenge. Where pastoralist communities are concerned, one might argue that the cyclical and often climate-driven nature of these conflicts should make them predictable and easy to prevent or manage. However, pastoral areas are characterized by a minimal and, in some cases, absent state presence and uncoordinated state responses have left stabilized communities vulnerable to their cross-border neighbors. Consequently, a course correction is urgently needed to return functionality to the system. This report advises USAID to assist in this course correction. Because the CEWARN conflict and early warning system is currently largely non-functional, there is little formal connection between the CEWARN Secretariat and the relevant CSOs. However, prior to mid-2014, strong links existed within the system between the CEWARN Secretariat, CEWERUs, and CSOs. In fact, many member states’ local conflict early warning systems were actually designed and initiated by the CSO/NGO community and later adopted by CEWARN. In addition, many of the ex-field monitors were employees of the CSOs and in most member states the CSOs were largely responsible for testing the system. In some instances, the CSOs were involved in the development of CEWARN’s 2012 - 2019 Strategy Framework. Both the CEWARN Secretariat and external donors appear to have overlooked the instrumental role of the CSOs. In designing the new system, the CEWARN Secretariat proposed changes to CSOs’ substantial and roles, which resulted in alienating many of these organizations. The report includes 12 recommendations that address both general and specific issues identified in the course of the evaluation. Recommendations 1. USAID should conditionally renew its assistance to the IGAD secretariat, the IDDRSI PCU, ICPAC, and CEWARN subject to these agencies’ satisfactory compliance with the other recommendations of this report. 2. As soon as possible, all IGAD agencies receiving USAID assistance should initiate comprehensive and coordinated core business planning exercises in line with the approach utilized by the PREPARED project, including all of the following steps:  Clear identification of an agency’s priority clients in the region;1  Clear identification of the products and services that are most needed and, therefore, most highly valued, by high-priority clients;  Clear identification of which of these products and services are satisfactorily provided by others in the region;  Clear identification of any remaining gaps in products or services related to an agency’s core capacities;  Clear identification of real-world constraints that may limit an agency’s mandated roles, powers or aspirations; and  Specification and monitoring of robust performance and outcome indicators, including specific indicators that measure coordination between agencies.  USAID, working individually or with like-minded groups or external donors, should 1 Note that external donors are not regarded as direct clients, rather they are supporters of service provision to regional clients. This keeps the system focused on outcomes, as is of most benefit to external donors. viii support these exercises. 3. All stakeholders should review their understanding of how best to utilize IGAD agencies, especially in relation to correcting misconceptions regarding IGAD’s real-world capabilities and limitations. (A cursory examination of which is provided in Annex 10 of this report.) 4. When external donors are considering funding positions within IGAD agencies, they should request specific information on potential limitations in IGAD’s ability to support and maintain the positions in the longer term and IGAD agencies should transparently provide such information. Where necessary, external donors should consider alternate models to boosting IGAD capacity, including the “assistance with outsourcing” or “consultant development” models outlined under Question IDDRSI/PCU 1. 5. The CEWARN Secretariat, with assistance from USAID, should urgently redesign the proposed “new” regional conflict early warning and response system. This revised system must incorporate the lessons learned from the pre-existing system and should consider new means of addressing funding issues, such as the timely liquidation and accounting of expenditures by the retargeting of external donor funds intended to support truly rapid responses to the country level.2 To avoid misconceptions, the evaluation team recommends renaming the Rapid Response Fund held by the CEWARN Secretariat to the Response Assistance and Research Fund. The revised system should also consider incorporating a methodology to expand CSO involvement, but only in ways that are consistent with the lessons learned to date (detailed direction provided in Annex 12). 6. In accordance with recommendations two and five above, the CEWARN Secretariat should add a medium-term boundary to its scope and consider developing a set of guidelines or a code of conduct for conflict early warning and response for eventual member state ratification. All agencies likely to be involved in responses to conflicts under any CEWARN-supported system would be subject to these guidelines, which also should include clear delineation of the legitimate roles of CSOs in such responses and discourage potential excesses by armed players. The guidelines should specifically address expanded measures for ensuring the safety of field monitors. 7. When providing assistance or advice at a regional level, USAID/Kenya and USAID/EA should remain cognizant of the intent and expected outputs of concurrent bilateral interventions in similar fields to prevent contradictory and inefficient results. For example, support to regional early conflict warning and response mechanisms should not encourage removal of field monitors while bilateral projects are in the process of actively training them. 8. All IGAD agencies assisted by USAID, particularly the IDDRSI PCU, should use the core business planning approach specified in recommendation two to guide regional knowledge management approaches. This includes moving to metadata storage where needed and playing more active roles in the promotion of regional data sharing where feasible. 9. External donors, including USAID, should monitor the success and application of the core business planning approaches specified in recommendation two. If satisfactory progress is made in focusing and monitoring the IGAD agencies’ activities, then USAID and other external donors should consider providing funds to cover the duration of such core 2 With possible member state in-kind matching requirements applied. ix business plans, using staged allocations contingent on results from performance/outcome indicators. 10. External donors, including USAID, should use all means at their disposal to encourage member states to commit to attending at least one ordinary meeting of the Council of Ministers annually. 11. As a matter of urgency, IGAD should compile and publish the detailed progress made against its 2003 strategy, preferably before or concurrently with adoption of any new strategy. Agency-Specific Recommendations IDDRSI  To prevent the loss of staff and thus capacity in the future, donors may wish to consider other models of assisting IGAD agencies with capacity building. If external capacity relevant to an agency’s need already exists in the local area, assistance may be given to engage external service providers to fill capacity gaps. If local external capacity does not exist, persons recruited to fill such gaps could be groomed to provide ongoing consultancy services once donor funding ends.  The IGAD Secretariat (incorporating the IDDRSI PCU) should focus on storing only carefully selected raw data, such as information generated internally through M&E processes or required to generate key products and services identified in core business planning. The secretariat should also design a complementary data structure that references external data sources, as this is a more efficient and achievable option that also prevents much of the workload associated with updating externally-sourced datasets.  IGAD should review the attributes, experiences, and education necessary for the knowledge management coordinator position and should determine a realistic package of benefits necessary to recruit and retain a qualified individual. IGAD will need to negotiate with USAID for the funding of this position. ICPAC  USAID should expand PREPARED’s role in conducting business planning for member state meteorological agencies to include ICPAC. CEWARN  The CEWARN Secretariat should identify clear and measurable key performance indicators for itself and facilitate CEWERUs’ application of aligned indicators.  The CEWARN Secretariat should focus its institutional strengthening efforts on CEWERUs most in need of support in the target countries of Djibouti, South Sudan, and Somalia, to improve these countries’ early warning infrastructure. It also should support IGAD-level monitoring of the implementation of the South Sudan peace agreement.  The CEWARN Secretariat should develop best practice guidelines for cross-border responses to conflict.  The CEWARN Secretariat should revitalize its early warning function by reengaging field monitors and urgently implementing changes to financial accountability systems to ensure external donor funds are correctly expended in a timely manner. x  While the new system had good intentions, there is urgent need for the CEWARN Secretariat to develop an alternative system as a compromise. This heavily revised new system must incorporate the lessons learned from the previous system, but should consider identifying ways to expand CSO involvement, consistent with the lessons learned (detailed suggestions provided as Annex 12). 1 1. Introduction 1.1 Intergovernmental Authority on Development (IGAD) The Intergovernmental Authority on Development (IGAD) replaced the Intergovernmental Authority on Drought and Development (IGADD), which was established in 1986 by the then drought-afflicted Eastern African countries of Djibouti, Ethiopia, Kenya, Somalia, Sudan, and Uganda. The State of Eritrea was admitted as the seventh member at the 4th Summit of the Heads of State and Government in Addis Ababa, Ethiopia, in September 1993. Although IGADD was originally conceived to coordinate the efforts of member states to combat drought and desertification, it became increasingly apparent that the authority provided a regular forum where leaders of Eastern African countries were able to tackle other political and socioeconomic issues in a regional context. Realising this, the Heads of State and Governments of Djibouti, Eritrea, Ethiopia, Kenya, Sudan, and Uganda, at an extraordinary summit on April 18, 1995, resolved to expand the mandate of IGADD and made a declaration to revitalise IGADD and expand cooperation among member states. (See Annex 1: Map of participating countries). The revitalised IGADD was renamed the Intergovernmental Authority on Development (IGAD).3 The organization’s mission and vision, as articulated on their web site, are included below. The IGAD Mission: “The IGAD mission is to assist and complement the efforts of the Member States to achieve, through increased cooperation:  Food security and environmental protection;  The promotion and maintenance of peace and security and humanitarian affairs; and,  Economic cooperation and integration.” 4 The IGAD Vision: “IGAD will be the premier regional organization for achieving peace, prosperity and regional integration in the IGAD region. The objectives of IGAD are to:  Promote joint development strategies and gradually harmonize macro-economic policies and programmes in the social, technological and scientific fields;  Harmonize policies with regard to trade, customs, transport, communications, agriculture and natural resources, and promote free movement of goods, services, and people within the region;  Create an enabling environment for foreign, cross-border and domestic trade and investment;  Achieve regional food security and encourage and assist efforts of Member States to collectively combat drought and other natural and man-made disasters and their natural consequences;  Initiate and promote programmes and projects to achieve regional food security, sustainable development of natural resources and environmental protection, and to encourage and assist the efforts of Member States to collectively combat drought and other natural and man-made disasters and their consequences;  Develop and improve a coordinated and complementary infrastructure in the areas of transport, telecommunications and energy in the region;  Promote peace and stability in the region and create mechanisms within the region for the prevention, management and resolution of inter-State and intra-State conflicts through dialogue;  Mobilize resources for the implementation of emergency, short-term, medium-term and long-term programmes within the framework of regional cooperation; 3 Agreement Establishing the Inter-Governmental Authority on Development (IGAD) 4 http://igad.int/index.php?option=com_content&view=article&id=93&Itemid=153&limitstart=1 2  Promote and realize the objectives of the Common Market for Eastern and Southern Africa (COMESA) and the African Economic Community;  Facilitate, promote and strengthen cooperation in research development and application in science and technology.” 5 The U.S. Agency for International Development East Africa Mission (USAID/EA) provided direct financial support to IGAD through a Limited Scope Grant Agreement (LSGA) that ran from September 24, 2006, through December 31, 2015. USAID/EA’s support to IGAD focused on three programmatic areas: 1. The Conflict Early Warning and Response Mechanism (CEWARN): USAID/EA supported the CEWARN Secretariat through its Horn of Africa Support Program from 2001 to 2004. From September 24, 2006, through December 31, 2015, USAID/EA supported the CEWARN Secretariat and its related Conflict Early Warning and Response Mechanism Units (CEWERUs) with the development and implementation of CEWARN’s 2012 to 2019 Strategy Framework through LSGA No. 623-0009.02. Total funding was $5,269,560. 2. The IGAD Drought Disaster Resilience and Sustainability Initiative (IDDRSI): Beginning in November 2012, with funding of $3,000,000, USAID/EA supported implementation of the IDDRSI strategy through the Platform Coordinating Unit (PCU) to build IGAD’s institutional capacity and to strengthen IDDRSI’s knowledge management system. 3. The IGAD Climate Prediction and Application Center (ICPAC): Beginning in March 2013, USAID/EA supported ICPAC through the Planning for Resilience in East Africa project under the Policy, Adaptation, Research and Economic Development (PREPARED) program. ICPAC has three priority areas: 1) institutional strengthening, 2) production of relevant geospatial climate applications and products tailored for use within East Africa, and 3) strengthening the interface between climate change information producers and end users. Funding for these activities was $250,000. 1.2 IGAD Conflict Early Warning and Response Mechanism To institutionalize conflict early warning and response capabilities, the IGAD Secretariat established the CEWARN Secretariat in 2001. Since 2006, the CEWARN Secretariat activities have been funded through an LSGA between USAID/EA and IGAD. IGAD/CEWARN and the CEWERUs seek to facilitate local, national, and intergovernmental cooperation and responses to cross-border conflicts in the Horn of Africa. With the IGAD member states’ adoption of the 2012 to 2019 Strategy Framework, CEWARN’s focus expanded dramatically from coordinating responses to pastoral conflict to integrating timely conflict early warning and response analysis into policy making at the local, national, and regional levels. CEWARNs work is transitioning from focusing exclusively on pastoralism to other sectors, including economic growth, environmental protection, governance, security, social welfare, and early warning response. It is important to note that USAID does not currently fund these new CEWARN sectors. Thematic areas have emerged within these sectors, including, but not limited to, election-related violence, land, inter-ethnic relations, and climate change. USAID is supporting the CEWARN Secretariat to upgrade the mechanism’s systems to fulfill this new mandate and to achieve the objectives set forth in the strategic framework. The CEWARN head office is in Addis Ababa. Its national level units, the CEWERUs, are located in seven IGAD countries: Kenya, Ethiopia, South Sudan, Sudan, Somalia, Uganda, and Djibouti. Each CEWERU brings together government, civil society representatives, and local committees to implement national conflict early warning and response systems. 5 http://igad.int/index.php?option=com_content&view=article&id=93&Itemid=153&limitstart=1 3 1.3 IGAD Drought Disaster Resilience Sustainability Initiative (IDDRSI) In September 2011, in response to the catastrophic phenomenon of recurrent droughts and related environmental concerns in a sustainable manner, IGAD heads of state convened a summit in Nairobi and made a collective decision to create IDDRSI. They called for the urgent introduction of strategies, policies, and programs that would strengthen member state and regional investments aimed at building resilience to future climatic and economic shocks. To drive IDDRSI forward, IGAD member states and their development partners agreed to form a Regional Drought Resilience Platform, which brings together member states, the IGAD Secretariat, and development and implementing partners including United Nations (UN) agencies, civil society, and specialized research and training institutions. As part of its institutional arrangements, the platform includes a General Assembly of participating stakeholders, a Platform Steering Committee, and a Platform Coordinating Unit (PCU). The IDDRSI platform provides the modalities for discussion of the region’s priorities and the possibilities for intervention by effected countries and development partners in support of drought resilience. It also provides an effective mechanism for coordinating the implementation of the drought resilience initiative. Under the IDDRSI platform, the PCU was created and embedded in the IGAD Secretariat in Djibouti. The PCU engages in regular technical and functional contact with IGAD member states and partners to: create awareness; plan, execute, monitor, and evaluate projects, including the development of country program frameworks and project identification and preparation; provide training and capacity building; prepare reports on program needs and progress; organize regional technical, policy, and coordination meetings; provide technical support as needed; and link activities on the ground with the IGAD Secretariat, Regional Platform, and interested partners. USAID supported two main IDDRSI activities: building a knowledge management system and institutional strengthening of the IGAD Secretariat through staff support in administration, finance, and procurement. 1.4 IGAD Climate Prediction and Application Center ICPAC is a specialized IGAD institution with eleven member countries6 and a shared mission to foster sub-regional and national capacity for climate information, prediction products and services, early warning, and related applications for sustainable development in the region. In each partner country, ICPAC has a designated lead and end-user liaisons who provide computer services and data management, climatology, climate diagnostics, prediction models, climate applications, documentation, research and development. ICPAC works with the national meteorological and hydrological services organizations within its member states and with the World Meteorological Organization and other research institutions to address regional climate-related challenges and risks, including climate change. ICPAC is also working to develop capacity for long-term climate change projections, but its specialty in seasonal predictions is most relevant for drought and disaster prevention. Through PREPARED, USAID/EA supports the center’s institutional strengthening, specifically in the area of early warning, predictions and their related applications. 6 The IGAD countries plus Rwanda, Burundi, and Tanzania. 4 2. Evaluation Objectives and Questions International Business and Technical Consultants, Inc. (IBTCI) received a task order from USAID on July 2, 2015 through the Evaluation Services and Program Support (ESPS) contract, to evaluate the effectiveness and sustainability of USAID/EA-supported IGAD activities. Prior to the receipt of the task order, the ESPS team worked with USAID to design a practical methodology that would enable the evaluation team to answer the evaluation questions while also balancing cost and time resources. 2.1 Evaluation Objectives This goal of this external performance evaluation was to help determine which programs and activities of IGAD and its institutions (CEWARN, IDDRSI, and ICPAC) are working well and why, and which have not achieved the expected results and why. It should also provide USAID/EA with information to guide modifications and mid-course corrections, if necessary, and will help IGAD, CEWARN, IDDRSI, and ICPAC improve their performance and subsequent results. Intended users include the IGAD Secretariat and IGAD institutions including: the CEWARN Secretariat, the IDDRSI Platform, and ICPAC; the IGAD Institutional Strengthening Action Plan (ISAP) Coordination Group; IGAD member states; USAID/EA; USAID/Washington (particularly the Resilience Secretariat) and USAID/EA Bilateral Missions (especially Kenya, Ethiopia and Somalia). 2.2 Evaluation Questions USAID divided the evaluation questions into general and cross-cutting themes and as specific questions to be answered for each activity/program. (See Annex 2: Statement of Work.) Overall 1. To what extent do the various IGAD efforts supported by USAID coordinate to achieve IGAD’s overall objectives? How could these efforts better complement each other to achieve IGAD’s overall objectives? 2. To what extent are current donor structures/relationships conducive to IGAD achieving its organizational objectives and how can these be improved? What are the primary areas USAID should focus on to effectively advance IGAD’s agenda? 3. To what extent are the CEWARN, IDDRSI and ICPAC activities sustainable? What is the extent of IGAD member states’ financial, political and in-kind support for these activities and how can that support be strengthened? 4. What has been IGAD’s experience with its USAID partnership? Are there areas that can be improved and if so, which areas are those? Program Specific CEWARN 1. To what extent is CEWARN on track to fully implement its strategic framework? What has worked well and what has not? Are all CEWERUs equally effective in advancing the strategic framework or are there varying levels of success? What factors contribute to any differing levels of success and what lessons can CEWERUs learn from each other? 2. Are CEWARN and its national level CEWERU interventions meaningful and responsive to cross-border conflicts that occur in the region? If so, what factors have made that possible? What have been the main challenges for CEWARN and the CEWARUs in addressing cross￾border conflicts? To what extent can CEWARN be expected to effectively contribute to advancing peace and stability in the East Africa region? 3. To what extent do civil society organizations (CSOs) in the East Africa region work with and support CEWARN and/or the CEWERUs? What recommendations do the regional 5 stakeholders, including CSOs, have for improving the effectiveness of the CEWARN and CEWARU programs? IDDRSI 1. How has IGAD leveraged USAID support to strengthen the PCU to implement the IDDRSI strategy? 2. Since its inception, how has USAID support developed the foundations for knowledge management in IGAD? What challenges have been experienced by IGAD in operationalizing the knowledge management system? How have these challenges been handled? 3. USAID/EA has supported the institutional capacity strengthening of IGAD by providing resources for IGAD to hire core finance and administration staff. To what extent have the expected results and/or outcomes been achieved? What steps is IGAD taking to assume responsibility for funding of these staff to avoid dependence on USAID? What is the anticipated timeline for IGAD to fully assume funding responsibility for these staff? ICPAC 1. To what extent has ICPAC been effective and efficient in achieving its strategic objectives and responding to climate change priorities? 2. To what extent are USAID-supported programs responding to thematic priorities as outlined in ICPAC’s strategic plans and other regional strategies agreed to by member states? 3. How are ICPAC’s current systems and processes helping or hindering the agency from meeting regional climate change demands and priorities? 3 Methodology and Limitations 3.1 Approach to Field Work ESPS established a two-person team to conduct the evaluation with a team leader and a senior investigator. (See Annex 3: Evaluation Team Member CVs.) Prior to initiating field work, there was a two-week planning phase (August 3 to August 14) to carry out the majority of the document review and conduct the team planning meeting. At this time, the methodology was finalized, instruments were developed, the mobilization of target respondents was finalized, and other field logistics were completed. This information is attached in Annexes 5 to 8. Field interviews took place from August 14 to October 12, 2015. 3.2 Site Selection The evaluation team collected data from stakeholders within institutions and organizations working with IGAD in five capitols: Djibouti, Djibouti; Addis Ababa, Ethiopia; Nairobi, Kenya; Kampala, Uganda; and Juba, South Sudan. 3.3 Target Groups and Selection Methods USAID/EA provided ESPS with key informants and the evaluation team analyzed IGAD’s documents to identify other important stakeholders and additional interviewees. (Please see Annex 8) To ensure homogeneity within respondent groups and to guide the organization of data and instruments, the stakeholders were then sorted according to their respective mandates into five categories: 1) the IGAD Secretariat (including finance/administration); 2) conflict prevention and management; 3) drought resilience and sustainability; 4) climate prediction and application; and 5) development partners or organizations. 6 To help ensure a rich source of data, the evaluation team carefully selected the key informant interview (KII) respondents based on their roles and responsibilities in IGAD’s USAID/EA-supported programs. 3.4 Data Collection Methods The evaluation team used a mixed methods approach to data gathering, including: 1) document review—primarily progress reports and statistical information available from IGAD and from other relevant reports and findings; 2) quantitative data received directly from relevant organizations through formal (written) information requests defining the specific types of data sought;7 and 3) qualitative data gathered through KIIs. (See Annex 4: Matrix of Questions and Sub-Questions.) 3.4.1 Document Review The document review provided the evaluation team with background information on IGAD’s activities and was critical in helping the team shape the approach to the evaluation. (See Annex 5: Complete List of Documents Reviewed.) It is worth noting that documentation, particularly documentation covering recent years, was scant and requests for additional documents went mostly unanswered. 3.4.2 Formal Information Requests In recognition that reporting and documentation related to IGAD activities was either scant or difficult to obtain, which would severely limit availability of some quantitative data, the evaluation team sent written requests to all relevant IGAD agencies and associates for additional information. These requests were sent out in the second week of the two-month evaluation fieldwork period and focused on obtaining official information regarding progress against strategies or plans (including related budget data). Requests were intentionally limited to information that should be easily obtainable from a basic M&E system and thus had the dual use of discovering information and testing the capacity of existing M&E systems. Even after extensive re-prompting, responses to these requests remained minimal. (See Annex 6: Responses to Information Requests.) 3.4.3 Key Informant Interviews The evaluation team conducted 106 KIIs with 47 female and 59 male respondents (both as individuals and in a group setting). The KIIs questions were designed to elicit information pertinent to the relevant evaluative questions. A specific subset of questions was asked of each respondent based on their roles and associations with relevant IGAD activities. (See Annex 7: Instruments. Also see Annex 8: List of Respondents for the KIIs.) In recognition that documentary evidence, particularly officially recognized quantitative data, was scarce, the evaluation team provided a numerical analysis of KIIs, as feasible. This analysis is provided in Annex 9, and is intended to demonstrate that the positions adopted by the team were not selectively drawn from favored key informants, rather were based on the majority of responses triangulated across countries and informant types. 3.5 Data Management To prevent data loss or other biases from occurring with the KIIs, notes were transcribed into data storage templates (in Excel) as soon as possible after interviews were conducted. These data were stored in a manner that disassociated raw response data from identifiers that would link data to an individual respondent. This helped to maintain respondent confidentiality. 7 Such formal information requests were kept modest in scope, and limited to data that could be reasonably expected to be easily extracted from any basic M&E system. Despite this, responses to these information requests were limited. 7 3.6 Ethical Considerations8 Respondents were given the option to refuse to answer particular questions (or to refuse the whole interview) if they believed a response would contain sensitive information. This was particularly important when discussing conflict-related matters with the national agencies of member states. Respondents were assured that their responses would not be traceable to them as individuals, but would either be reported as summaries or unattributed examples. Only members of the evaluation team had access to the transcripts and raw data. While respondent identifiers will be redacted from the raw dataset provided to USAID/EA at the end of the evaluation process, general categorization information (organization type, geographical location, position type, gender, etc.) will remain linked to the raw response data. This report is a synthesis of the team’s analysis drawn from the interviews of numerous respondents. Any quotes provided to highlight particular issues are not attributed to an individual by name.9 The evaluation team certified that they had no conflict of interest in undertaking this evaluation. These signed statements are stored at the ESPS office in Nairobi and are available for review upon request. 3.7 Limitations There were some limitations associated with the methods chosen for this evaluation. They include: Quantitative data: Statistics and records maintained by IGAD and other relevant agencies were taken at face value. While the evaluation team attempted to triangulate among the different data sources (e.g., documents and KIIs), efforts to fully validate specific details was beyond the scope of the evaluation. Minimizing bias: The evaluation team used multiple mechanisms to minimize respondent and interviewer bias: 1. KIIs were implemented using standardized guides rather than a detailed interview format that might force respondents to answer questions about projects of which they had no knowledge. 2. Recall bias: The KII questions and sub-questions focused on IGAD’s current activities and operations during the evaluation period and included the previous three years (2012 to September 2015). 3. Interviewer bias was mitigated by convening daily team debriefs, rolling data analysis, and rapid transcription of in-depth interview data from key informants. 4. Social desirability bias was mitigated by standard confidentiality assurances and by structured questioning that specifically elicited both strengths and weaknesses. Sampling: Given that a list of preferred KII respondents was predetermined, this purposive sample needs to be considered as inclusive rather than statistically representative. While the team attempted to extend the inclusiveness of the original list whenever feasible, it was not possible to retrospectively create a statistically valid sampling frame. Findings of the report, therefore, are provided on this basis and no statistical inferences can or should be made. While not ideal, this situation is common in assessments focused on organizations. Given the high diversity in the roles of the individuals employed by organizations such as IGAD, randomization, stratification, or clustering of sampling has little real-world significance and the only truly representative sampling approach would be to interview the entire population. 3.8 Analysis The evaluation team used an inductive approach to analysis focusing on the emergent themes and issues arising from the raw KII data. These themes and issues have been coded and analyzed in 8 IBTCI: Ethical Standards and Protocols for Field Research 9 IBTCI: Ethical Policy Guidelines 8 respect to various analytical domains, particularly the evaluative questions specified in the task order. In preference to more generic analysis facilitated by use of off-the-shelf qualitative analysis software, in recognition of the specific information requirements of the evaluative questions, the evaluation team designed respondent questions accordingly. Many respondent questions, therefore, inquire about relevant “issues raised” in the forms of strengths or weaknesses of current practices or situations. A spreadsheet-based system was used to conduct a partial strengths, weaknesses, opportunities, and threats (SWOT) analysis for each relevant question. A numerical analysis of KII data was conducted to support basing all major findings on the entire dataset and not just selective interpretations. The analysis is based on the partial SWOT analysis, where respondents were asked to raise issues related to IGAD and its agencies (via specific instrument questions). Where the data meaningfully allows, triangulations across organization type and countries are also provided. It is important to understand that this analysis is issues-based (not individual, respondent-based). Given that a purposive sampling method was used, no statistical significance can be inferred. (See Annex 9: Numerical Analysis of KII data.) Because the team received few responses to formal information requests, analysis of statistical trends was impeded. However, the key findings of the report stemming from the KIIs are considered to reflect the highest priority issues pertaining to the evaluative questions. 4 Findings, Conclusions, and Recommendations 4.1 Introduction Because this is partially an evaluation of institutions, findings related to key strengths or weaknesses are often cross-cutting and relevant to a range of evaluation questions. 10 This evaluation addresses each evaluation question separately. However, lengthy discussions of common findings under each question would have been repetitive. Therefore, to mitigate such repetition, findings relevant across each IGAD agency are presented separately. Where conclusions are made from more than one finding, the findings are noted in the conclusions. Some recommendations are also cross-cutting. When a recommendation is specific to a finding or conclusion, it is provided immediately following the conclusion. When a recommendation is cross￾cutting, to avoid duplication, the team has added a recommendation section that carefully references the cross-cutting recommendation and its related evaluation findings and conclusions. This keeps the report concise, logical, and easy to read. 4.2 IGAD Accomplishments Article 7 of the agreement establishing IGAD lists the organization’s 11 specific aims and objectives, which expand its earlier mandate to address drought and desertification to include tackling political and socioeconomic issues in a regional context.11 Further explanation of IGAD’s mandate is found in its mission statement: “The IGAD mission is to assist and complement Member States to achieve through increased cooperation: food security and environmental protection; promotion and maintenance of peace and security and humanitarian affairs; and economic cooperation and integration.” In its 2011 to 2015 Strategy Implementation Plan, IGAD spells out its four pillars, each with specific program areas. The table below captures IGAD’s current progress per strategic pillar, as reported by respondents during field work. 10 Or key strengths and weaknesses in relation to the operational environment 11 Agreement Establishing the Inter-Governmental Authority on Development (IGAD) 9 Table 1: Progress against each strategic pillar Strategic Pillar Progress to Date i. Agriculture, natural resources, and environment Program areas: a. Agriculture, livestock, and food security b. Natural resources management c. Environmental protection d. Climate variability and change e. Applied research and civil society organizations support  Launch of the Resilience Analysis Unit  Trans-boundary projects, e.g., Ethiopia-South Sudan Biodiversity Project  Climate information analyzed and shared within the region ii. Economic cooperation and integration and social development Program areas: a. Trade, industry, and tourism development b. Infrastructure development c. Health and social development  On-going infrastructure projects, e.g., Ethiopia-Djibouti; Kenya-South Sudan￾Ethiopia port and road construction  Developed IGAD regional sustainable tourism master plan  Customs streamlining policies iii. Peace and security and humanitarian affairs Program areas: a. Conflict prevention, management, and resolution b. Political affairs c. Security sector d. Humanitarian affairs  Mediating South Sudan and Somalia conflicts  IGAD Security Sector Program created to combat violent extremism and trans-border crime  Regional CEWARN functioning well until recently iv. Corporate development services Program areas: a. Institutional strengthening and capacity building b. Research, science, and technology  Recruited 11 finance and administrative staff members to the PCU  Recruited a Knowledge Management Coordinator, but the position is now vacant 10 4.2.1 Evaluation Question 1 To what extent do the various IGAD efforts supported by USAID coordinate to achieve IGAD’s overall objectives? How could these efforts better complement each other to achieve IGAD’s overall objectives? Evaluation Question 1a: To what extent do the various IGAD efforts supported by USAID coordinate to achieve IGADs overall objectives? IGAD Objectives (Source: IGAD Regional Strategy, Revised Edition, 2013) IGAD Secretariat IDDRSI ICPAC The CEWARN Secretariat 1. Promote joint development strategies and gradually harmonize macro-economic policies and programs in the social, technological, and scientific fields Contributes at all levels Contributes in relation to resilience fields Contributes in relation to forecasting science and technology Contributes in relation to cross￾border conflict related fields 2. Harmonize policies with regard to trade, customs, transport, communications, agriculture, and natural resources and environment, and promote free movement of goods, services, and people within the region Contributes at all levels Contributes in relation to agriculture, natural resources, and environmental fields Contributes in relation to agriculture, natural resources, and environmental fields Contributes in relation to agriculture, natural resources, and environmental fields 3. Create an enabling environment for foreign, cross-border, and domestic trade and investment Contributes at all levels NA NA NA 4. Initiate and promote programs and projects to achieve regional food security and sustainable development of natural resources and environmental protection, and encourage and assist efforts of member states to collectively combat drought and other natural and man-made disasters and their consequences Contributes at all levels Contributes at all levels Contributes in relation to natural (climatic) disasters Contributes in relation to man￾made (conflict) disasters 5. Develop and improve a coordinated and complementary infrastructure, in the areas of transport, telecommunications, and energy in the region Contributes at all levels NA NA NA 6. Promote peace and stability in the region and create mechanisms within the region for the prevention, management, and resolution of inter-state and intra-state conflicts through dialogue; Contributes at all levels Contributes in relation to natural resource conflicts Contributes in relation to natural resource conflicts Contributes at all levels 7. Mobilize resources for the implementation of emergency, short-term, medium-term, and long￾term programs within the framework of regional cooperation Contributes at all levels Contributes at all levels Contributes at all levels Contributes at all levels 8. Facilitate, promote, and strengthen cooperation in research development and application in science and technology Contributes at all levels Contributes in relation to resilience fields Contributes in relation to forecasting science and technology Contributes in relation to cross border conflict related fields 9. Provide capacity building and training at regional and national levels Contributes at all levels Contributes in relation to resilience fields Contributes in relation to forecasting science and technology Contributes in relation to cross border conflict related fields 10. Generate and disseminate development information in the region Contributes at all levels Contributes in relation to resilience fields Contributes in relation to forecasting science and technology Contributes in relation to cross border conflict related fields 11 Coordination There is a strong stakeholder perception that IGAD agencies have attempted to expand their own profiles at the expense of linkages with the rest of IGAD (i.e., “empire building”).12 Many agencies who see themselves as relatively independent from their parent organization may regard raising their own profile as an end in itself.13 This implies that separate entities within IGAD are preoccupied with building their own prestige and recognition, making them reluctant to coordinate with each other. Respondents noted a potential concern: close and effective coordination between agencies risks diluting external visibility, which in turn reduces any direct credit they might receive for an achieved (and shared) outcome.14 The classic irony of this “empire building” problem is that increases in coordination have the potential to increase the magnitude of outcomes. Thus, the share of credit going to each entity involved also would increase.15 Further evidence of poor coordination between IGAD agencies can be seen in documentation requirements. The strategic and other planning documents of IGAD agencies are often cross￾referenced, but with the exception of the IDDRSI platform, they are not clearly interdependent. For example, CEWARN and ICPAC strategic plans do not require cooperation with any other IGAD agencies to claim successful progress against their respective strategies.16 Conclusion While IGAD has many structures and periodic forums in place that could facilitate close cooperation, in practice, cooperation among the IGAD platforms and programs directly supported by USAID remains low. Comprehensively addressing this challenge would require a major restructuring of IGAD. Requiring more explicit definitions of success (e.g., indicators) that specifically encompass such interdependence may encourage IGAD agencies to actively seek expanded cooperation. The responses to the program-specific questions began to answer the question of whether IGAD has met its objectives in regard to CEWARN, IDDRSI, and ICPAC. The evaluation team must concede some difficulty, however, in answering the broader question regarding whether IGAD, in general, has met the objectives put forth in their current 2003 strategy. Accessing information regarding progress towards these objectives proved difficult, despite the team making formal written requests early in the evaluation period after finding that the reports provided for their review and relevant websites held limited information. These requests were intentionally constrained primarily to ‘progress against plan’ data to ensure they were not too onerous and the requested data could be easily extracted from a basic M&E system. Despite a number of written follow-ups, the requested information was not provided. Therefore, the team can only conclude that either the IGAD M&E systems were not able to perform to this basic task or IGAD was reluctant to release such information. While the team recognizes the independence of IGAD as an organization, we also note that external donors remain IGAD’s primary source of operational funding. 17 Donors are increasingly reluctant to continue funding if past outcomes are not readily apparent.18 It is in IGAD’s interest to urgently address this matter and a perfect opportunity is currently available. With completion of its 2003 strategy and the upcoming introduction of a revised strategy (now in draft form),19 compilation and publication of progress made against the 2003 strategy would be a sensible option for highlighting achievements to date, and may assist in ensuring continued external funding. 12 KIIs with multi-lateral agencies Kenya, September 2015; KII Member State Agency, Ethiopia, September 2015 13 Ibid 14 KII, multi-lateral agency, Kenya, September 2015 15 IASC CLUSTER APPROACH EVALUATION, 2ND PHASE, APRIL 2010 16 CEWARN Strategic Framework pg. 19 (Strategic Outcomes); The Protocol on the Establishment of IGAD Climate Prediction and Applications Centre (ICPAC) Strategy, August 2010 (Objectives) 17 KIIs, IGAD Secretariat, Djibouti, August 2015 18 KII, External Donor, Kenya, September 2015; KII USAID, October 2015 19 KIIs, IGAD Secretariat, Djibouti, August 2015 12 Meeting Objectives The IGAD Secretariat has delivered some valuable high-level outputs (e.g., mediation of the South Sudan internal conflict and facilitation of the economic infrastructure project)20 and multiple stakeholders believe that IGAD platforms and agencies currently funded by USAID are able to deliver unique and important regional products and services.21 These positive perceptions support IGAD’s strong status in the region. No other organization can claim the member state ownership or focused convening powers that IGAD has, especially for this particularly troubled part of Africa. Its formal committees range from permanent secretaries to heads of state.22 Its ability to maintain a focus on issues in the region gives it an advantage over larger entities, such as the African Union, particularly in relation to facilitating negotiations to solve problems facing the region.23 Conclusion Provided its real capabilities and constraints are recognized, IGAD is a valuable vehicle for advancing regional reforms and for applying standardized systems to address common problems. These systems, ideally, will continue to break down the barriers represented by country borders. Moreover, IGAD’s ability to focus on issues solely in this region gives it an advantage over larger groups such as the AU, particularly in relation to facilitating the negotiation of solutions to problems in the region. Challenges in Meeting Objectives Funding IGAD and its agencies face a high level of funding uncertainty because funding for IGAD programs comes from member states, external donors, and IGAD itself, and is thus unpredictable.24 Under the agreement establishing IGAD, annual member state contributions are meant to be determined by IGAD’s Council of Ministers at “ordinary” meetings. In recent years, no ordinary meetings have been held. As a result, contributions were made on an ad-hoc basis concerning the amount and the timing of the contributions.25 These shortfalls or delays have marked effects on all IGAD activities, especially when IGAD agencies receive funding late in the implementation cycle.26 External donors usually commit funds based on their own funding cycles, which vary from donor to donor and with the types of assistance provided. Typically, the funding cycle is five years or less between renewals or renegotiations. All forms of donor assistance, including trust funds, require legal agreements that specify how funds must be used and how to account for expenditures. This means donor dependent agencies, such as IGAD, must constantly juggle disparate cycles of external assistance to ensure some constancy in overall levels of funding.27 In addition, provision of funds is often staged with later tranches dependent on meeting certain milestones or conditions,28 which compounds the difficulties IGAD agencies encounter in receive timely funds. IGAD agencies are dependent on the performance of their implementing partners over which they have varying degrees of control and thus cannot always predict when certain milestones or conditions will be met.29 Conclusion The availability of funds from IGAD’s potential sources (member states and external donors) is 20 KII, IGAD Secretariat, Djibouti, August 2015; South Sudan: Keeping the Faith with the IGAD Peace Process. Africa Report No 228, July 2,2015; International Crisis Group, page 1; The Sudan – IGAD Peace Process, Signposts for the way forward; ISS Paper 86, March 2004, pp.1-2 21 KII, CSO/NGO, South Sudan, September, 2015; Finland’s Support to Institutional Strengthening of IGAD, mid-term evaluation pp 13-22; The IDDRSI Strategy, IGAD 2013 p 14; USAID East Africa. Evaluation of IGAD/Conflict Early Warning and Response Mechanism (CEWARN) for USAID/EZ/RCMG 2009 22 Article 8. AGREEMENT ESTABLISHING THE INTER-GOVERNMENTAL AUTHORITY ON DEVELOPMENT (IGAD) 23 KII, IGAD Secretariat, South Sudan, September 2015; Francis, D. (2006) Uniting Africa: Building Regional Peace and Security Systems, Ashgate Publishing, UK. p. 231; The IDDRSI Strategy, IGAD, 2013 p. 14 24 KII, IGAD Secretariat, Djibouti, August 2015; Interviews with CSO/NGO, South Sudan, September 2015 25 KIIs: IGAD Secretariat, Djibouti, August, 2015; CEWARN, Ethiopia, August, 2015 26 KIIs CWARN and CEWERU staff, August and September 2015 27 KIIs with IGAD Secretariat, Djibouti, August 2015 28 KIIs, IGAD Secretariat, Djibouti, August 2015; USAID, Kenya, October 2015; IGAD Organizational Structure Review, GiZ, June 2006 pp. 12-16 29 Ibid 13 generally restricted within the region. This situation has little chance of significantly changing in the near future. This unpredictability of funding likely has a negative impact on IGAD’s ability to plan or implement specific programs. While the establishment of trust funds is one approach already used to mitigate the unpredictability of funding in the short term, this does not address funding issues in the longer term. This is because funds are not likely to be provided unless IGAD incorporates financial accounting systems that comply with donor requirements. Any approach to resolve the uncertainty of funding must recognize that IGAD and its agencies have limited potential sources of income. Client Capture To overcome resource constraints, IGAD agencies often rely heavily on direct assistance from national agencies in the member states where they are headquartered. These close ties have caused stakeholders to perceive the agencies as biased, believing they focus assistance activities in host member states.30 This perception of “client-capture” is most evident in the CEWARN Secretariat and ICPAC, especially since they are well removed from the IGAD Secretariat in Djibouti. There does appear to be some basis for these perceptions. For example, substantial back payments were provided to the terminated CEWARN field monitors in Ethiopia, where the CEWARN Secretariat is hosted, whereas requests for similar payments in other countries went unheeded.31 However, such client capture is not entirely negative in effect. In fact, it is likely that the key regional successes of ICPAC to date would not have occurred had it not been for the strong assistance directly provided by the University of Nairobi and the Kenyan Meteorological Bureau.32 Conclusion Both perceptions and manifestations of client capture of IGAD agencies by member states may discourage other organizations from regarding them as productive partners that will coordinate activities at a regional level. A distinction must be made, however, between client capture of a regional IGAD agency, and the domestication of the systems, structures, and actions promoted by that regional agency within member states. Domestication of such systems, structures, and actions is possibly the best outcome achievable for any regional program—even when the domestication includes some loss of higher level control by (and credit for) the regional agency. Domestication of such systems generally improves both resourcing and sustainability and demonstrates member state ownership and a commitment to program objectives. Work Planning While the USAID-funded activity, PREPARED, is providing assistance primarily to ICPAC to promote and facilitate appropriate service provision, stakeholders see room for performance improvement, especially in service provision. KII respondents raised a total of 85 issues related to IGAD planning, 87 percent of which were respondent-identified as weaknesses. One reason for these perceived weaknesses is that they believe that the agencies lack focused and realistic medium￾term planning that incorporates adequate M&E systems. Insufficient M&E results in poor tracking and outcome-level accountability related to expenditures and has created a growing frustration or reluctance among stakeholders to partner further with IGAD agencies.33 In addition, the lack of a service provision approach in planning is directly evident from documentation. For example, eight of the nine strategic pillars of the CEWARN Strategic Plan relate primarily to organizational objectives, rather than the outcomes of the agency’s work.34 Conclusion Actions and documentation suggest that IGAD and its agencies have not consistently undertaken work planning that effectively identifies and focuses on a realistic and well-defined core business, including provision of key products and services. 30 KIIs with member state agencies, Kenya and Ethiopia, September 2015 31 KII member state agency, Kenya, October 2015. (It should be noted that the evaluation team could not confirm this, as the financial information requested of CEWARN was not provided. 32 KIIs member state agency, Kenya, October, 2015; multi-laterals, Kenya, 2015 33 KIIs with multi-lateral donors, including USAID staff September 2015 34 CEWARN Strategic Framework, pg. 21 (Strategic Outcomes) 14 Procurement and Accounting Systems The procurement and accounting systems that IGAD uses to administer expenditures of external donor funds either include duplicative workloads or risk not complying with the needs of various donors.35 Improvements in this area relate to currently duplicative financial procurement and financial reporting. IGAD and its agencies must account for the use of donor funds according to the requirements of both IGAD and its donors’ accounting systems. Given IGAD’s range of donors, its financial staff must be familiar with and accommodate a diverse number of accounting requirements. Attempts have been made to develop a single financial system that would satisfy all internal and external requirements, but obtaining acceptance of any single system by all donors has been difficult.36 Conclusion Current financial reporting structures are inefficient. However, because donors must answer to their own internal auditors, this may well be an intractable problem. Achieving improvements will require compromises on behalf of both IGAD and its external donors. Attempts made to date have been shallow. Targeted consultations and a renewed attempt to harmonize requirements among IGAD donors is likely to produce more constructive results, especially if negotiations and system development specifically include the donors’ auditing staff. Stakeholder Perceptions Multiple stakeholders appear unaware of the limitations that IGAD faces, especially concerning IGAD’s agencies’ independence, IGAD’s neutrality, and its ability to enforce compliance or regional initiatives within member states.37 In particular, it appears some stakeholders are unfamiliar with Article 6A of IGAD’s Establishment Agreement, which states: “The member states solemnly reaffirm their commitment to the following principles: 1) The sovereign equality of all member states; 2) Non-interference in the internal affairs of member states.” 38 These constraints impede IGAD’s current roles well before the limits of constitutional agreements are reached.39 While it was outside the scope of this evaluation to do so, the evaluation team compiled the misconceptions that appear most prevalent. (See Annex 10: Stakeholder Misconceptions.) As it was outside the scope of this evaluation, targeted or systematic data collection for this purpose was limited. Care was taken in drafting this annex to remain as objective as possible, however, the content of the annex should be regarded as the evaluation team’s opinions. Evaluation Question 1b: How could these efforts better complement each other to achieve IGAD’s overall objectives? This question speaks to the recommendations provided by the evaluation team. Because recommendations are cross-cutting, touching on all the evaluation questions, rather than repeating the same recommendations, the response to this question is answered in the Recommendation Section 5.0 of this report. 4.2.2 Evaluation Question 2 To what extent are current donor structures/relationships conducive to IGAD achieving its organizational objectives and how can these be improved? What are the primary areas USAID should focus on to effectively advance IGAD’s agenda? 35 KII IGAD Secretariat, Djibouti, August 2015; IGAD Organizational Structure Review, GiZ, June 2006. pp. 12-16. 36 KII IGAD Secretariat, Djibouti, August 2015; Implementation Review of the Horn of Africa Strategic Framework, Council of the EU General Secretariat, Political and Security Committee, February 2013. p16. 37 KIIs IGAD Secretariat, Djibouti, August 2015, Multi-lateral agencies, Kenya, September 2015 38 KII, multilateral agency, Kenya, September 2015. : KII, IGAD Secretariat, Djibouti, August 2015; Article 6A. AGREEMENT ESTABLISHING THE INTER-GOVERNMENTAL AUTHORITY ON DEVELOPMENT (IGAD) 39 The IDDRSI Strategy, IGAD, 2013 p. 14; KIIUSAID activity, Kenya, September 2015; Hassan, R., (2013) CEWARN’s new strategy framework; Implications for Sudan and South Sudan’s existing and emerging conflicts, African Security Review 22.2, June 2013, p.26; KII, Multilateral Agency, Kenya, September 2015 15 Evaluation Question 2a: To what extent are current donor structures/relationships conducive to IGAD achieving its organizational objectives and how can these be improved? As noted above, the unpredictability of funding is a key issue affecting IGAD and its agencies’ abilities to achieve their objectives. An associated issue is developing between IGAD and its external donors, which is unpredictable, short-term, and/or preferential funding by external donors that impedes IGAD and its agencies’ ability to define and adhere to a focused work plan. This lack of focus, in turn, discourages and frustrates donors. They do not see clear medium-term objectives, nor clear and measurable indicators of success. Some of the donors interviewed reported that these measures are now mandatory prerequisites for funding.40 The related inability of IGAD and its agencies to account for funds already provided through outcomes or in financial records is something that external donors cannot continue to overlook.41 IGAD agencies, in cooperation with external donors, have established “basket funds,” such as the Joint Financing Agreement (JFA) and the CEWARN Rapid Response Fund (RRF), with the intent of creating improved funding certainty.42 However, if documentation regarding past funding is not forthcoming, financial controllers are still required (sometimes by legislation) to halt further funding.43 A perception exists among external donors in particular that a general absence across all IGAD agencies of robust M&E systems that encompass financial accounting where relevant has led to significant replenishment issues.44 The CEWARN RRF has faced this problem for a number of years; no new money has been provided since 2012.45 Conclusion IGAD’s (and particularly the CEWARN Secretariat’s) inability to account for the expenditure of funds through programmatic outcomes or through financial accountability is something that external donors cannot continue to overlook. Recipient agencies must remain acutely aware that donors have their own internal auditing requirements that are beyond the ability of most donor staff to change. These requirements are not something that donors can easily make exceptions for and organizations that continue to ask for exceptions will continue to face funding issues. Recommendation The response to this problem must be systemic. The evaluation team believes the most productive step would be for IGAD and its agencies to adopt the core business planning approach used by the USAID-funded PREPARED activity. The resulting highly-focused and realistic work plans would allow simpler and more transparent M&E, including definition of clear and measurable outputs, outcomes, and performance indicators. This would provide IGAD with a means to communicate concrete and effective achievements. Moreover, it would clearly communicate the specific intent of IGAD programs to donors so that areas of overlapping interest could be more precisely identified. Such core business planning inherently includes approaches to streamline bureaucratic processes within an organization so office systems and the work plan complement each other. Recommendations 2-5, 7-8, 10, and 12 in Recommendation Section 5.0 also directly or indirectly apply to this question. Evaluation Question 2b: What are the primary areas USAID should focus on to effectively advance IGAD’s agenda? While it is recognized that donors should be reluctant to risk further funding of agencies that cannot account for past expenditures, all the agencies currently funded have produced valuable regional products and services. The evaluation team consistently found that stakeholders perceived that the 40 KII, external donor, Kenya, September 2015; KII USAID, October 2015; IGAD Organizational Structure Review, GiZ, June 2006. pp. 12-16 41 KIIs with USAID staff September/October 2015; IGAD Organizational Structure Review, GiZ, June 2006. pp. 12-16 42 KII, IGAD Secretariat, Djibouti, August 2015 43 KIIs, USAID, Kenya, October 2015; IGAD Organizational Structure Review, GiZ, June 2006. pp. 12-16 44 KIIs: CEWARN Ethiopia, August 2015; External Donor, Kenya, September 2015; IGAD Organizational Structure Review, GiZ, June 2006. pp. 12-16 45 KII USAID, October 2015. The evaluation team requested documentation for this report however it was not received. 16 IGAD platforms and agencies currently funded by USAID are able to deliver unique and important regional products and services, despite these entities’ varying levels of current performance. Other non-regional organizations lack the member state ownership or convening powers that make it possible to negotiate truly regional solutions. Given the interconnectedness of conflict, resilience, and climate change issues across the region, country-by-country solutions are less likely to be effective than regional mechanisms. Other regional organizations lack a focus on key Horn of Africa member states or they are so large that negotiation of shared positions on issues specifically related to IGAD member states would be unlikely. Additionally, any attempts to replicate IGAD activities to date would likely confuse and impede existing progress. Conclusion IGAD’s platforms and agencies have the potential to deliver products and services region-wide. If USAID’s objectives related to conflict, resilience, and climate change in the IGAD region remain a high priority, then IGAD and its relevant agencies are well positioned to continue this work. This suggests that, subject to a number of caveats, USAID should retain its current focus on funding IDDRSI, CEWARN, and ICPAC. These caveats are addressed under responses to more specific questions. They primarily include more focused planning across all IGAD agencies to enable more focused monitoring and reporting of outcomes, and a course correction for CEWARN that recognizes its operational constraints regarding rapid reinstatement of a functional conflict early warning system. Recommendation Recommended changes in USAID’s focus within the currently-funded IGAD agencies and platforms are dealt with under program specific evaluative questions. Recommendations one through ten in Recommendation Section 5.0 are pertinent to the response to this question. The evaluation team believes that if the key recommendations for reform (closely tied to regional service provision) are enacted, many current issues may be rectified. 4.2.3 Evaluation Question 3 To what extent are the CEWARN, IDDRSI, and ICPAC activities sustainable? What is the extent of IGAD member states’ financial, political, and in-kind support for these activities and how can that support be strengthened? Evaluation Question 3a: To what extent are the CEWARN, IDDRSI, and ICPAC activities sustainable? Scarcity and unpredictability in resource availability remain key issues for IGAD agencies. Member state annual contributions are considered the most relevant source of funding for the sustainability of IGAD activities. Issues related to sustainability of these contributions are discussed in detail in response to the next question (IGAD Question 3 B), but at this point it should be noted that external donors remain IGAD’s main source (in the quantity) of operational funds. 46 When considering resource responsibilities transferred to entities outside of the IGAD system, the picture has been varied. National meteorological agencies have been heavily involved in “data￾resourcing” of ICPAC since its inception and Kenyan meteorological expertise has heavily underpinned ICPAC human resources.47 IDDRSI, as a platform, is almost entirely reliant on member state government agencies, NGOs in member states, or other regional/multilateral organizations (e.g., CGIAR, ILRI, UNDP, WFP, FAO, UNICEF, etc.) for implementation of on-the-ground 46 KII external donor, Kenya, October 2015; IGAD Secretariat, Djibouti, August 2015; Institutional Assessment of the Intergovernmental Authority for Development (IGAD), Assessment Report, Assessment Report, Delegation of the European Commission to Ethiopia Authority for Development (IGAD), July 2006. pp. 34-35. 47 KIIs member state agencies, Djibouti and Kenya August/September 2015; ICPAC Organizational Self-Assessment, ICPAC, 2010, p. 12 17 activities.48 In the area of resilience, actions have been conducted in accordance with the country performance papers (CPPs) and a “transferal” of responsibility for resourcing has taken place. Paradoxically, while the CEWARN Secretariat systems are almost completely dysfunctional, they provide the most impressive and concrete examples of successful transfer of resourcing responsibility, with both member state funding and complete domestication of systems occurring.49 While formal member state financial commitments and external donor funding may not be strong contributors to the sustainability of IGAD activities at this point in time,50 it should be noted that the impact of ad-hoc (project or system-level) resourcing of IGAD activities by member states can be considerable if the right products and services are delivered.51 In some cases, systems or products have been so well accepted and ingrained within member state agencies and with other stakeholders that even (perhaps misplaced) IGAD attempts to remove them have been strongly resisted.52 Conclusion As of the completion of this evaluation, the institutional elements of IGAD programs are not self￾sustaining. However, to the extent that the products and services provided by these programs have been valued by key regional clients, domestication and adoption of standardized regional systems and products has been considerable and does yield some sustainable results. The evaluation team concludes that the sustainability of these activities is largely dependent on three factors: 1. The internal availability of resources necessary for the implementation of activities; 2. The rate that the responsibility for the provision of resources is transferred to others; and 3. The relevance of products and services produced by these activities to the needs of key regional clients. Recommendation Recommendations two through five, seven, nine, and ten found in Recommendation Section 5.0 also directly or indirectly apply to this question. Evaluation Question 3b: What is the extent of IGAD member states’ financial, political and in￾kind support for these activities and how can that support be strengthened? In-Kind Contributions In-kind contributions by member states to IGAD agencies have been more forthcoming compared to annual, formal financial contributions. ICPAC’s regional and seasonal forecasting is a mainstay of many national meteorological agencies, but it is the member state agencies that routinely provide the raw data upon which ICPAC forecasting is primarily based.53 All member states now have IDDRSI CPPs in place. This took considerable internal effort and coordination to achieve among the national agencies.54 The in-kind support for and domestication of CEWARN systems that took place prior to mid-2014 demonstrated strong support for use of these systems. Other measures of political commitment can be seen in the heightened activity of the ministerial and state-level committees convened by IGAD. In recent years, they have held over 60 extraordinary meetings to deal with important regional issues, including the most recent internal conflict in South 48 KIIs IGAD Secretariat, Djibouti, multi-lateral agencies, Kenya, August/September 2015 The IDDRSI Strategy, IGAD, 2013 pp. 22-24 49 See Annex 11, Lessons Learned 50 Discussed fully in Question IGAD 6 51 See responses to program specific evaluative questions. 52 See Annex 11, Lessons Learned 53 KIIs, member state agency, Djibouti, August 2015; Member State Agency Uganda, September 2015; ICPAC Organizational Self-Assessment 2010, page 15 54 KIIs IGAD Secretariat, Djibouti, August 2015; External donor, Ethiopia, September 2015; IDDRSI Program Report March 2015 18 Sudan.55 However, while these meetings were being held, no ordinary meetings of the ministerial￾level committees were taking place.56 This difference is interesting because these ordinary meetings are where housekeeping issues—including the discussion and approval of budgets—are decided.57 Conclusion The member states’ political commitment to IGAD remains high in spite of their generally poor, formal commitment to provide annual funding to the Secretariat. Financial Contributions There is no question that a serious problem exists with both the timeliness and completeness of annual member state financial contributions to IGAD. 58 This results in many issues, including those associated with the expansion of IGAD and its agencies’ permanent staff. 59 The first priority for use of member state funds obtained by each IGAD agency is the payment of staff salaries and staff￾related administrative costs, including executive travel. Servicing these priorities often leaves little or funding for these agencies’ operational budgets. External donors remain IGAD’s primary source of operational funds.60 This external funding can be used to fund temporary, project-based positions within IGAD agencies, but donors are often reluctant to directly support staff by building their technical capacities if they will not be absorbed as permanent staff. 61 The member states appear to have a weak commitment to the timely delivery of annual financial contributions to IGAD. On the other hand, member state commitments to IGAD appear quite strong from a service uptake or political perspective. When IGAD presents member states with useful products, services, or opportunities the members readily respond and support IGAD.62 For example, some member states agreed to coordinate infrastructure development regionally. This is driving important regional development, such as infrastructure development projects. 63 It also is being used as a launching pad to improve bilateral ties among member states. The distinction between annual financial support and political or in-kind support is an important one: though some member states actively avoid delivering their formal, financial contributions, they may still greatly value the benefits they get from being an IGAD member, and they may actively participate in the delivery of regional gains.64 The AU instituted a solution to address this lack of annual payments within their membership. If contributions are not made, voting and speaking rights can be withheld. 65 This may work for organizations with a large number of members, but for smaller regional groups, such as IGAD, this rule would likely exclude the participation of key players in identifying and applying regional solutions. This is especially pertinent concerning high-level or sensitive regional issues. IGAD cannot afford to lose key members’ ownership of solutions, as often all members are key. Conclusion As noted throughout this report, lack of consistent funding creates crippling issues for IGAD. Shortfalls and unpredictability in member state contributions place a very strong constraint on 55 KII IGAD agency, Ethiopia, September 2015; IGAD Secretariat, Djibouti August 2015; USAID staff, Kenya, October 2015 56 Ibid 57 Article 10. AGREEMENT ESTABLISHING THE INTER-GOVERNMENTAL AUTHORITY ON DEVELOPMENT (IGAD) 58 The evaluation team requested financial documents from the Secretariat noting member states contribution levels as well as timeliness of contributions however, these documents were not provided to the team. 59 Article 6A. AGREEMENT ESTABLISHING THE INTER-GOVERNMENTAL AUTHORITY ON DEVELOPMENT (IGAD); Institutional Assessment of the Intergovernmental Authority for Development (IGAD), Assessment Report, Assessment Report, Delegation of the European Commission to Ethiopia Authority for Development (IGAD), July 2006. pp34-35 60 KIIs: Member states staff; (Institutional Assessment of the Intergovernmental Authority for Development (IGAD), Assessment Report, Assessment Report, Delegation of the European Commission to Ethiopia Authority for Development (IGAD), July 2006. pp34-35 61 KIIs USAID staff, Kenya October 2015 62 KIIs CEWARN Ethiopia, ICPAC Kenya, August/September 2015 63 Ibid; KII IGAD Secretariat, Djibouti, August 2015: KII Member State, Djibouti, August 2015 64 KIIs CEWARN Ethiopia; ICAPAC Kenya, August/September 2015 65 KII, IGAD Secretariat, Djibouti, August 2015; Constitutive Act of the African Union Article 23.2, July 2000 19 sustainable expansion of human capacity within the organization and its agencies. Despite the lack of consistent and sustainable financial support, the evaluation team believes that meeting annual contributions is a poor indicator of member states’ political commitment to IGAD and its programs. What appears to be happening is more along the lines of a “tragedy of the commons,” not member state apathy towards IGAD. The issue of how to overcome current problems associated with non￾compliance with annual funding commitments will be difficult to resolve. Member state agencies consulted were reluctant to discuss the reasons behind late or absent payments, not least because such decisions were made at higher levels than the agencies operationally engaged with IGAD. A first step may therefore be to add some transparency to discussion of these issues. A vital step in achieving this transparency is the conduct of ordinary meetings of the Council of Ministers, at which budgets and other administrative issues are discussed. For reasons that remain somewhat unclear, instituting these ordinary meetings has been extremely lax over the past several years (see also response to Evaluative Question 1a). Recommendation Recommendations three and eleven in Recommendation Section 5.0 directly or indirectly apply to this question. 4.2.4 Evaluation Question 4 What has been IGAD’s experience with its USAID partnership? Are there areas which can be improved and if so, which areas are those? USAID is considered one of IGAD’s most important partners.66 Respondents lauded USAID’s leadership when the Global Alliance for Action for Drought Resilience and Growth was founded in 2012. Through the Global Alliance, IGAD has been linked to other international donors and USAID was credited with facilitating these linkages. 67 USAID has also supported building ICPAC’s capacity in weather prediction.68 Additionally, the role played by the United States Government in partnering with IGAD and South Sudan during the mediation process was extremely important.69 USAID’s support to IGAD is diverse. The secretariat, IDDRSI, ICPAC, and the CEWARN Secretariat are all beneficiaries of USAID funding for objectives related to institutional strengthening and other program/project goals.70 IGAD appreciates USAID’s leverage in the donor community and considers their partnership important and strategic as a regional organization.71 This viewpoint hinges on IGAD’s expectation that USAID will use its influence to assist in coordination of donor support to IGAD in line with the Paris Declaration on Aid Effectiveness.72 USAID’s influence could be helpful given the challenges IGAD faces in dealing with multiple donor reporting frameworks, multiple institutional assessments, and the time implications of responding to individual donor demands.73 Some respondents also had concerns about disbursement of funds to IGAD and its specialized platforms and agencies. Of the $1.6 million in USAID funds provided to the CEWARN Secretariat, IGAD has only received $500,000.74 This has resulted in cash flow challenges that have negatively affected the implementation of CEWARN’s strategy.75 Another disbursement-related challenge is the spacing of tranches of the payouts. ICPAC was concerned that the change from quarterly to monthly 66 KIIs: CEWARN staff, Ethiopia, August 2015; KIIs Member State Agency Staff, Djibouti, August 2015 67 KIIs: IGAD Secretariat, Djibouti, August, 2015; External Donor, Kenya, October 2015 68 Group interview with ICPAC staff, Kenya, September 2015 69 Follow up interview with IGAD Secretariat, August 2015 in Djibouti 70 KIIs: IGAD Secretariat, CEWARN and ICPAC, August – September 2015 71 KIIs: IGAD Secretariat and Member State Agency, Djibouti, August 2015 72 Ibid 73 Ibid 74 KII CEWARN staff; CEWARN, Ethiopia, August 2015 75 Ibid 20 fund disbursements would mean more time spent liquidating funds.76 The framework of funding, which is routed through IGAD where ultimate accountability lies, has in effect punished one IGAD agency for the liquidation failure of a sister agency. For example, when IDDRSI delayed liquidating its funds, money to the CEWARN Secretariat was withheld.77 The CEWARN Secretariat’s quandary is worse: it has to wait for satisfactory financial reports from its beneficiary CEWERUs before finally liquidating USAID funds.78 Respondents raised concerns about how long it takes to make funding decisions. Between September 2013 and June 2014, the CEWARN Director had more than 15 meetings with USAID, spending much time in 2014 on planning. However, USAID approval only came in January 2015.79 Conclusion USAID remains one of IGAD’s core development partners, not just for funding, but also for USAID’s leverage and facilitative role in the donor community. Key areas of improvement focus on the timely disbursement of funds. Recommendation Recommendations two, seven, eight, and ten in Recommendation Section 5.0 directly or indirectly apply to this question. 4.3 CEWARN80 CEWARN is a collaborative effort of the seven IGAD member states and one of IGAD’s programs targeted at mitigating and preventing violent conflict in the sub-region. Since its establishment in 2002, CEWARN has been functioning with a particular focus on cross-border pastoralist and related conflicts. CEWARN's vision is embodied in its tagline: "Empowering stakeholders to prevent violent conflicts.’’ Its mission is to establish itself as an effective and sustainable sub-regional mechanism for conflict early warning and response, while fostering cooperation among relevant stakeholders to respond to potential and actual violent conflicts in the IGAD region and to contribute to the peaceful settlement of disputes in the sub-region. Its mandate is ''To receive and share information concerning potentially violent conflicts as well as their outbreak and escalation in the IGAD region, undertake analysis of the information and develop case scenarios and formulate options for response.'' Through its national network of governmental and non-governmental stakeholders, CEWERUs, National Research Institutes (NRIs) and Field Monitors (FMs), CEWARN undertakes its conflict early warning and response function in three clusters: the Karamoja Cluster (covering the cross-border areas of Ethiopia, Kenya, South Sudan, and Uganda); the Somali Cluster (covering the cross-border areas of Ethiopia, Kenya and Somalia), and the Dikhil Cluster (covering the cross-border areas of Djibouti and Ethiopia). 4.3.1 CEWARN Evaluation Question 1 To what extent is CEWARN on track to fully implement its Strategic Framework? What has worked well and what has not? Are all CEWERUs equally effective in advancing the Strategic Framework or are there varying levels of success? What factors contribute to any differing levels of success and what lessons can CEWERUs learn from each other? 76 Group interview, ICPAC staff, Kenya, September 2015 77 KII CEWARN staff, Ethiopia, August 2015 78 KIIs: CEWARN staff, Ethiopia; CSOs/NGOs, Uganda, August-September 2015 79 KII CEWARN staff, Ethiopia, August, 2015 80 http://www.cewarn.org/index.php?option=com_content&view=article&id=51&Itemid=53 21 Evaluation Question 1a: To what extent is CEWARN on track to fully implement its strategic framework? The CEWARN strategy identifies nine pillars.81 The first seeks to expand the agency’s coverage of the typologies, the causes of violent conflict, and the geographic areas of focus. This pillar is reflective of a regional consensus among CEWARN’s stakeholders to expand the geographical scope of coverage in line with member states’ priorities. The second pillar addresses organizational and network capabilities aiming to improve CEWARN’s structures through training and skills transfer. The third pillar speaks to network engagement and communications. It focuses on expanding networks to embrace new institutions that will improve CEWARN’s visibility in the region. The fourth pillar addresses research and analysis to inform early warning and concrete response efforts. The fifth pillar supports and scales up response initiatives to improve interventions’ response time. The sixth strategic pillar addresses learning to identify impact and make adjustments through an M&E system that employs participatory approaches. The seventh pillar aims at encouraging member states to increase their financial and in-kind contributions and to cultivate new sources of funding. The eighth pillar emphasizes data quality, a strong and secure data bank, and skilled (analytical) staff. The ninth pillar addresses the quality of decision-making focusing on ensuring evidence-driven decisions concerning peace and security issues in the region. The following table provides an overview of the country-specific progress or issues regarding each of the nine pillars. Table 2: Progress by country against their strategic pillars82 STRATEGIC PILLAR COUNTRY PROGRESS Djibouti Ethiopia Kenya South Sudan Uganda 1. Expanding the coverage of the typologies, causes of violent conflict, and geographic areas of focus Unable to meet with CEWERU team in Djibouti. No data available. Intending to expand: Received equipment for establishing Situation Rooms in 10 zones (in 9 regions). Conflict mapping & assessment done (awaiting Somalia & South Sudan to finish). For the last 4 or 5 months, system not working. Already expanded coverage to whole country. National conflict mapping done. System working. Expansion to some states hampered by political strife. Have procured some equipment to some states, but some were looted in Upper Nile during conflict. Expanding coverage beyond northeastern Uganda. Setting up a situation room. System not working (no data). 2. Organizational and network capability Confirmed: getting training in conflict analysis, M&E, and exchange visits. Confirmed: participating in and offering trainings on conflict analysis, M&E, and conflict sensitive journalism. Also received country study tour delegations. Confirmed: receiving training in conflict analysis and conflict early warning. Also participating in study tours. Confirmed: setting up School Peace Clubs and getting some training. 3. Network engagement and No data on new CSOs, but Not sure about quality control (for Have own peace monitors in The CEWARN Secretariat has Advertised for expressions of 81 CEWARN. 2012. The CEWARN 2012 to 2019 Strategy Framework. Pp. 21 – 26 82 Based on KIIs with CSO/NGOs, member state agencies and community members in Ethiopia, Kenya, South Sudan, and Uganda between August and September 2015 22 STRATEGIC PILLAR COUNTRY PROGRESS Djibouti Ethiopia Kenya South Sudan Uganda communications (getting new partners, etc.) CEWERU has members on its technical committee. new civil society actors in place of FMs). addition to FMs. Feel use of CSOs will be a challenge without training. identified some potential CSOs in South Sudan. CSOs are awaiting further communication. interest for CSOs to apply for 5 NRI roles. Only one applicant. 4. Research and analysis No data. Conflict mapping and assessment done. Data collection stalled. National conflict mapping County conflict profiles Success stories Documentation Data collection by UNDP-funded peace monitors ongoing Conflict assessment ongoing. Data collection by FMs stopped. Peacebuilding study report done, supported by CEWARN. Data collection by FMs stalled. 5. Supporting and scaling response initiatives to improve response time Acknowledged resolution of conflict between two cross-border communities in 2013. Concerned about slow implementation of peace dividend projects, particularly the Magado project. Concerned about slow disbursement of RRF. Evidence of use of early warning to inform response. Concerned about slow disbursement of RRF. Evidence of use of early warning to inform response. Concerned about slow disbursement of RRF. Evidence of use of early warning to inform response. 6. Participatory M&E and Learning No data. Received M&E training. Improved project reporting. Participated in exchange visits for learning. Received M&E training. Documented lessons learned. Hosted delegations on study visits. Received M&E training. Participated in exchange visits for learning. No data. 7. Financial and administrative resources and systems (encouraging member states to increase their financial and in￾kind contributions) Acknowledged in-kind support from Djibouti government. Federal and regional administrations are part of the response system (support is through response). Passed Peace Policy, a basis for formal budgetary support to CEWERU. Government supports response. County governments support Peace Committees. Federal and state administrations are part of the response system. In-kind support to CEWERU. National and local administrations are part of the response system. In-kind support to CEWERU. 8. Data quality No information. Evidence of stalled data collection. Independent data collection. Evidence of stalled data collection. Evidence of stalled data collection. 9. Decision quality No data. Questioned The CEWARN Secretariat decision on attempting new approach. Questioned The CEWARN Secretariat decision on attempting new approach. Questioned The CEWARN Secretariat decision on attempting new approach. Questioned The CEWARN Secretariat decision on attempting new approach. Conclusion At this point in time, the CEWARN Secretariat is not administering a functioning regional conflict early warning and response system and the implementation of its strategic framework must be regarded as stalled. 23 Evaluation Question 1b: What has worked well and what has not? What Has Worked Well: The CEWARN system has evolved over time. CEWARN was established in response to an IGAD decision made in 2000 as part of a broader peace and development mandate.83 Under IGAD’s Division of Peace and Security, CEWARN was activated when the Protocol on the Establishment of a Conflict Early Warning and Response Mechanism for IGAD member states was enacted in July 2003.84 In operationalizing this early warning and response mechanism, IGAD adopted a bottom up, process-oriented approach that built upon existing efforts, mechanisms, and skills within the sub￾region.85 CEWARN established a system of local information collection networks to gather and document data on cross-border and related pastoral conflicts. At the community level, field monitors were deployed under CEWARN supervision and CEWERUs were set up at the national level (with local committees managed by CEWERUs).86 Member state-initiated changes made to the original system were based on sensitivities in releasing data by member states and the ability of the CEWARN Secretariat to effectively provide rapid response assistance under the RRF. 87 These changes, however, were strongly based on the original system supported by the CEWARN Secretariat. An overwhelming number of respondents thought the original system was effective and efficient in its reliable collection of useful data and its sufficiently-rapid local delivery of appropriate responses. Prior to mid-2014, member states created their own national CEWERUs and utilized FMs, local peace committees, and national (usually nongovernmental) agencies to compile and analyze data (e.g., NRIs). They also used national (usually nongovernmental) agencies to administer external funding for relevant CSO projects (Financial Administration Units) designed to implement their conflict early warning and response systems. What Has Not Worked Well: Certain elements of the pre-2014 system were not considered sufficiently sustainable by some CEWARN Secretariat and donor personnel. Such elements included the use of paid “individual” FMs and the requirement of small numbers of monitors to cover large geographical areas. 88 While these perspectives are theoretically correct, the means of correcting these problems incorporated into the proposed new system (post 2014) did not take into account all real-world constraints. The new system also overlooked some positive aspects of the pre-2014 systems that evolved in member states. 89 For example, many CEWERUs had based their long-term national planning on the pre￾existing system; a significant change in the system, therefore, would negate these existing achievements.90 Respondents acknowledged that the CEWARN Secretariat’s strength lay in its cross-border work and understood that the system cannot reasonably be expected to address every kind of conflict in member states.91 Its plan to have an SMS-based early warning system was questioned as potentially raising national sovereignty concerns in the absence of safeguards.92 The system that existed prior to mid-2014 was not the system originally proposed by CEWARN in 2002. CEWERUs responded to these constraints by implementing remarkably consistent local solutions, which included local 83 CEWARN. 2006. CEWARN Strategy 2007 – 2011. p.13 84 Ibid., p. 3 85 Ibid., p.14 86 Ibid., p.16 87 Based on key informant interviews with CSO/NGOs, member state agencies and community members in Ethiopia, Kenya, South Sudan, and Uganda between August and September 2015; Early Warning in the Horn of Africa, Press Release, SaferWorld, August 2015 88 KII, CEWARN, Ethiopia, August 2015; and USAID group interview, Kenya, September 2015 89 KII, CSO/NGO, South Sudan, September 2015 90 e.g. Strategic Plan for Uganda Conflict Early Warning and Early Response Unit, Uganda CEWERU, 2012-2017. 91 Ibid 92 KII, member state agency, Uganda, September 2015 and Key Informant Interview, Member State Agency, Ethiopia, August 2015 24 community peace dialogues, traditional community pacts, and the use of cultural mechanisms for conflict resolution.93 The CEWARN Secretariat did not seem to be wholly averse to implementation of such local solutions and even supported the establishment of national or more local “situation rooms” in some member states. However, the CEWARN Secretariat’s support for CEWERU innovations was not consistent and was occasionally contradictory. 94 The CEWARN Secretariat depleted the funds it had to support the pre-mid 2014 system before fully accessing the funds it needed to implement or even test the proposed new system.95 The CEWARN Secretariat’s ability to access new funds has been seriously impeded because it has not been able to fully liquidate and account for previous disbursements according to the requisite donor standards. 96 The dominant perception of respondents was that ad-hoc attempts to begin implementation of the proposed new system have not been handled well and have alienated many important stakeholders, particularly interested CSOs.97 In addition, the decision to select five NRIs per country, where previously there was one per country, was not popular.98 Ex-FMs and ex-NRIs consistently voiced growing annoyance at the CEWARN Secretariat for continuing to try to extract information or services from them when their contracts have not been renewed for nearly a year and since they were terminated in a manner they found disrespectful.99 Introduction of the new conflict early warning system proposed by the CEWARN Secretariat has not appreciably advanced as of the end date of data gathering for this evaluation and the CEWARN Secretariat appears to have ignored the lessons learned during the evolution of the pre-existing system. One key lesson learned is that the CEWARN Secretariat requires much more cooperation and input of stakeholders at the local, national, and regional levels.100 (See Annex 11: Lessons Learned.) The proposed system is also counter to the existing CEWARN strategy and key stakeholders in member states, particularly CSOs and government agencies, are resistant to the new system to such a degree that proceeding with its implementation has become untenable.101 Even if the new system had progressed further, its design conflicts with the current CEWARN strategy, which other stakeholders have been using to coordinate their efforts with the CEWARN Secretariat. For example, the bilateral USAID SAFE program in Uganda will soon finish training over 500 conflict and early warning FMs to make them available to the national CEWERU.102 While concerns remain about the CEWERUs’ capacity to handle and support these trained FMs, the CEWERU is being advised by the CEWARN Secretariat that FMs are no longer required. It is not the intent of the evaluation team to suggest that the pre-2014 system was without flaws. Accounting for funds spent was difficult or impossible at all levels of the system (see also response to 2c) and safety issues have arisen for FMs (with at least one FM killed in Uganda).103 However, it is difficult to see how the changes currently proposed by CEWARN will not exacerbate rather than address these issues. Changes include even further dissolution of financial management roles and 93 Based on KIIs with CSO/NGOs, member state agencies and community members in Ethiopia, Kenya, South Sudan, and Uganda between August and September 2015 94 e.g. their reported reaction to Kenya CEWERU ‘fully domesticating’ CEWARN systems to monitor and respond to internal election violence. 95 KIIs CEWARN Ethiopia; external donor Kenya, August/September 2015; The evaluation team requested figures from CEWARN however, these were not provided. 96 Ibid; IGAD Organizational Structure Review, GiZ, June 2006. pp. 12-16 97 KIIs with CSO/NGOs and member state agencies in Ethiopia, Uganda, and Kenya) Early warning in the Horn of Africa, Press Release, SaferWorld, August 2015. 98 KIIs with CSO/NGOs and member state agencies in Ethiopia, Uganda, and Kenya 99 KIIs CSOs/NGOs, South Sudan, Uganda, Kenya, September 2015; KIIs community members South Sudan, Uganda, Kenya August/September 2015 100 Quoted in CEWARN. 2006. CEWARN Strategy 2007 – 2011. Addis Ababa: CEWARN Secretariat, p. 21 101 KIIs with CSOs/NGOs South Sudan, Uganda and Kenya September/October 2015; CEWARN 20012 to 2019 Strategy Framework, p. 29 102 KII USAID Uganda Staff, September, 2015 103 Based on KIIs with member state agencies in Uganda, September 2015. 25 identification of whole CSOs as field monitors.104 From a sustainability perspective the use of paid FMs is problematic, but the use of whole CSOs for this purpose, as proposed under current planning, is likely to be at least equally flawed because such CSOs will also require funding and other assistance to carry the work effectively. As with other emergency services, such as fire departments, some fixed operational costs may be unavoidable. The RRF managed by the CEWARN Secretariat fulfils an important function. It enables CSOs (through longer-term response efforts) to maintain peace through initiatives such as peace dividend projects, but it does not and cannot play a meaningful role in rapid response to conflict.105 Conclusion The CEWARN Secretariat is well placed to provide important products and services to the region, but its strategy and planning do not focus on a set of key roles and responsibilities that emphasize such service provision. Consequently, important products and services are not delivered, or in some cases are not even identified. The abrupt termination of the pre-mid-2014 system in favor of a new untested system supports the perceptions of member state stakeholders: that the CEWARN Secretariat has effectively dismissed all their contributions to date. At the time of this evaluation, the CEWARN Secretariat was not managing a functional conflict early warning and response mechanism (but it had been until approximately mid-2014). What remains are the remnants of CEWARN’s system, which is now primarily operated by member states. At this point in time it appears that an impasse has been reached: even if funds for implementing the new system are mobilized quickly, the alienation and resistance that has developed in relation to dismissing the locally evolved, pre-existing system will render effective implementation of the new system virtually impossible. Despite the CEWARN Secretariat’s claims of conducting extensive consultations, a majority of respondents, particularly FMs, maintain that they were not well informed about the termination of the pre-existing system in favor of a new system. This was particularly the case for some CSOs. After putting considerable effort into making the pre-existing system a broad success, the CSOs interviewed claim that they were not aware of the change until they saw the CEWARN Secretariat’s advertisements seeking new CSO partners in national media. While the new system simplifies both incident and situation reporting, the evaluation team also independently identified a number of concerning insertions in the proposed new system that we believe will greatly complicate the conflict early warning and response system in a manner that will promote a theoretical approach, rather than local functionality (see Annex 11). It is also of concern that even the simplification of raw data collection under the proposed new system remains reliant on responses to subjective, ordinal (rating) scales, rather than on verifiable criteria. Potential for inter-observer variation in responses to such subjective scales alone would suggest they are a questionable input into regional conflict-related analysis algorithms. The new system also fails to consider many constraints identified within the system prior to mid-2014. Finally, while it is recognized that the design of the proposed new system had good intentions, the evaluation team has concluded that re-establishment of a regional conflict early warning and response system under the CEWARN Secretariat will require significant modification and simplification of the proposed new design. Particular attention should be paid to the key constraints and other critical lessons identified in the pre-2014 system. Lessons learned are discussed in Annex 11 and detailed recommendations for a revised CEWARN system are provided in Annex 12. 104 Based on KIIs with CSO/NGOs, member state agencies, and community members in Ethiopia, Kenya, South Sudan and Uganda, August and September 2015. 105 Ibid. 26 Recommendation Recommendations two through ten in Recommendation Section 5.0 directly or indirectly apply to this question. Evaluation Question 1c: Are all CEWERUs equally effective in advancing the strategic framework or are there varying levels of success? Among the four CEWERU coordinators interviewed, they unanimously agreed that Kenya has been the benchmark for the rest. The Kenyan CEWERU technical committee members acknowledged having hosted country study teams (not only from other CEWERUs, but also delegations from Zimbabwe and Tanzania) that were interested in learning from Kenya’s experience with the system.106 Kenya has appointed peace monitors in areas not covered by field monitors and has peace committees countrywide. The country has recently passed a peace policy and a sessional paper giving effect to the policy.107 Thus, there is an expectation that there will be formal, budgetary support for the CEWERU’s activities. Previously, such assistance was limited to the remuneration of core staff, in-kind support, and the government’s response to conflict, which has been reactive rather than preventive and dependent on armed players such as police and military. 108 The CEWERU coordinators interviewed offered these updates:  The Ethiopian CEWERU is advancing its architecture to cover 10 zones in 9 of its 11 regions. However, in the last four or five months, the system has stalled due to financial limitations. 109  Uganda’s CEWERU observed that there has been no early warning data collection for more than eight months.110 The CEWERU is also looking to expand its coverage, especially given improving security conditions in Karamoja. “Unless the CEWERU expands its range of conflict assistance and reaches new areas, its relevance will fade,” said one Ugandan CEWERU member. 111  The South Sudan CEWERU, while having realized some achievements (See Table 3: Progress by country against their strategic pillars), has had difficulty in establishing structures in the hotspots of the civil strife (Jonglei and Upper Nile).112 The South Sudan CEWERU has not received less funding from the CEWARN Secretariat than Ethiopia, Kenya, and Uganda have received.113  Note: The evaluation team did not meet the Djibouti CEWERU. Their members were unreachable during the visit to the country. However, their response to the evaluation team’s information request shows that their most recent activity was in 2013. 114 Conclusion The country-specific CEWERUs have advanced the strategic framework with varying degrees of success. Recommendation The CEWARN Secretariat should apply clear and measurable key performance indicators for itself and should coordinate and facilitate CEWERUs to apply aligned indicators. Recommendations five through seven found in Recommendation Section 5.0 directly or indirectly apply to this question. 106 Group interview on 11th September 2015 in Nairobi, Kenya 107 KII, member state agency, Kenya, September 2015) (Uwiano Platform for Peace, Strengthening Coordination and Leadership Towards Peaceful Elections and Political Transition in Kenya, 2013, p.2) (Ministry of State for Provincial Administration and Internal Security, National Peace Policy on Peacebuilding and Conflict Management, 2012) 108 KII, member state agency, Kenya, September 2015 109 Based on a Discussion on 31st August 2015 in Addis Ababa, Ethiopia. It was corroborated in another discussion on 2nd September 2015 in Addis Ababa. 110 KIIs member state agency staff, Uganda, September 2015 111 KIIs, member state agency staff; CSO/NGO staff, Uganda, September 2015 112 KIIs, member state agency, CSO/NGO staff, South Sudan, September, 2015 113 KII member state agency, South Sudan, September 2015 114 Formal information request response, Djibouti CEWERU, September 2015 27 Evaluation Question 1d: What factors contribute to any differing levels of success and what lessons can CEWERUs learn from each other? Political stability, or lack thereof, in the different member states has influenced the CEWERUs’ success. For example, insurgency is a key factor explaining the South Sudan CEWERU’s inability to set up operational structures countrywide. In South Sudan’s Upper Nile state, rebels looted CEWERU equipment.115 The CEWERU in the Republic of South Sudan (as Africa’s newest state) is playing catch up, especially since the rest of the IGAD member states have been signatories since 2002. The CEWERUs’ successes are varied due to resource limitations and/or the availability of funds. Kenya’s CEWERU has flourished because of its ability to secure funding from different development partners. Through support from UNDP it established structures countrywide and disseminated peace monitors.116 The ability to hire peace monitors independent of the CEWARN Secretariat support has enabled Kenya’s early warning system to maintain its functionality despite the termination of the FM contracts. The Ethiopia CEWERU has had support from multiple sources as well, including UNDP, Pact, and GIZ.117 This support however, does not involve maintaining FM. 118 There is also shared sentiment that some CEWERUs get more resources from the CEWARN Secretariat than others do. South Sudan and Djibouti were cited as those benefiting least.119 A closely related factor is member states’ levels of support for their CEWERUs. Kenya’s CEWERU successfully lobbied for the adoption of a peace policy and a sessional paper to give it force of law. In this way the CEWERU will get annual budgetary allocations for its operations.120 Uganda’s CEWERU is in the process of drafting its peace policy.121 Finally, the extent of involvement of CSOs in CEWERUs also has affected the levels of progress. South Sudan’s CEWERU relied on the early warning infrastructure set up by Catholic Relief Services.122 Kenya’s progress, by the CEWERU’s admission, is also down to having significantly coopted CSOs in CEWERU programs and activities.123 Conclusion The varied levels of advancement among CEWERUs reflect each member state’s domestic factors, which either facilitate or hinder progress. Countries that have embraced broader civil society participation and diversified their sources of funding have realized more progress. Recommendation The CEWARN Secretariat should focus its institutional strengthening efforts on CEWERUs that need it most, particularly Djibouti, South Sudan, and Somalia, with the objective of improving the early warning infrastructure in these countries. The secretariat should also support IGAD-level monitoring of the South Sudan peace agreement. Recommendations five through seven in Recommendation Section 5.0 directly or indirectly apply to this question. 4.3.2 CEWARN Evaluation Question 2 Are CEWARN and its national-level CEWERU interventions meaningful and responsive to cross border conflicts that occur in the region? If so, what factors have made that possible? What have been the main 115 KII, member state agency, South Sudan September 2015 116 Group interview with member state agency staff and CSO/NGOs, Kenya, September 2015 117 KII, CEWARN staff, Ethiopia, August, 2015 118 KIIs, external donors, Ethiopia, Kenya, August - October 2015. 119 KII, member state agency, South Sudan, September 2015 120 Group interview with member state agency staff and CSO/NGOs, Kenya, September 2015 121 KII, CSO/NGO, Uganda September 2015 122 KII, CSO/NGO and member state agency, South Sudan, September 2015 123 Group interview with member state agency staff and CSO/NGOs, Kenya, September 2015 28 challenges for CEWARN and the CEWARUs in addressing cross-border conflicts? To what extent can CEWARN be expected to effectively contribute to advancing peace and stability in the East Africa region it serves? Evaluation Question 2a: Are CEWARN and its national-level CEWERU interventions meaningful and responsive to cross-border conflicts that occur in the region? If so, what factors have made that possible? In all four countries where CEWERU stakeholders were interviewed (24 interviews and/or group discussions), there is general agreement that previous CEWARN Secretariat and CEWERU interventions were relevant and meaningful. 124 The main types of cross-border conflicts cited were inter-community armed cattle raids and violent conflicts over water and pasture.125 The early warning function of the CEWARN mechanism has made different informed responses possible. In Uganda’s Karamoja Integrated Disarmament and Development Program, FMs’ reports were used in targeted weapons collection efforts by the Uganda People’s Defense Forces.126 Four of the five CEWERU stakeholders interviewed confirmed that the Karamoja region is now much more stable and cattle raids and firearms are significantly reduced.127 The Kenyan and Ethiopian CEWERUs implemented peace dividend projects on their respective sides of the border (in Sololo and Magado respectively).128 The rationale for the development of a market center is that local trade creates opportunities for meaningful, non-violent, intergroup interactions, thereby reducing the likelihood of violent conflicts. Such outcomes are to be monitored over time, but a significant reduction in incidents of violent cross-border conflict has occurred on the Kenya￾Ethiopia border based on CEWARN Secretariat/CEWERU data.129 There are, however, concerns that the peace dividend projects have taken too long to complete. 130 Based on interviewees’ responses, there was inter-community violence in the Nadapal-Lokichoggio corridor of the Kenya-South Sudan border in August 2015 that led to the government forces’ response. Raiders from the South Sudanese Toposa community allegedly killed a Kenyan police officer in the process. The Kenyan and South Sudanese local administrations then held a cross￾border meeting to ease the tensions. Three suspects were later arrested, convicted, and sentenced to one year in jail.131 Lower level, cross-border coordination between CEWERU structures has greatly facilitated local interventions across countries. The evaluation found that in some areas, there are cross-border peace committees, like the one on the Ethiopia-Kenya border.132 Factors facilitating these interventions are the local community peace structures as well as cross-border coordination between the member states’ local government structures. Such coordination is occurring on the borders of Kenya and Uganda; South Sudan and Uganda; Kenya and South Sudan; Ethiopia and Kenya; Ethiopia and South Sudan; and Djibouti and Ethiopia.133 Thus, the CEWARN mechanism appears to have provided a good framework for regional cross-border collaboration on conflict issues.134 Besides political goodwill for cross-border collaboration, resource factors, such as the existence of the RRF, have enabled the implementation of some cross-border peace dividend 124 KIIs, member state agencies, Ethiopia, Kenya, South Sudan, and Uganda, August and September 2015 125 KIIs/group interviews, community members, Ethiopia, Kenya, South Sudan, and Uganda, August - September 2015 126 KIIs, Uganda, September 2015 127 Ibid. 128 KIIs, member state agencies, CSO/NGOs, and community members, Ethiopia and Kenya, August - September 2015 129 KIIs, member state agencies and CSO/NGOs, Ethiopia, August - September 2015 130 KIIs, member state agency, Ethiopia, August 2015; NGO/CSO, Ethiopia, August 2015; community members, Ethiopia, August 2015 131 KIIs, community members and CSOs/NGOs, South Sudan, September, 2015 132 KII community members, Kenya, September, 2015 133 KIIs, member state agencies, community members, and CSOs/NGOs in Djibouti, Ethiopia, Kenya, South Sudan, and Uganda, August - October, 2015 134 KIIs, CSO/NGO staff, Kenya, September, 2015 29 projects. Conclusion CEWARN structures at the sub-regional, national, and local levels have facilitated coordination of cross-border interventions to some extent. The CEWARN mechanism has been useful for cross￾border information sharing and for coordination of cross-border responses. As to the responsiveness, armed state responses have been more reactive than preventive. Recommendation The CEWARN Secretariat should develop best practice guidelines for cross-border responses to conflict. Recommendations five through seven in Recommendation Section 5.0 directly or indirectly apply to this question. Evaluation Questions 2b: What have been the main challenges for CEWARN and the CEWARUs in addressing cross border conflicts? The recent stalling of CEWARN’s early warning system due to the sudden reorientation of the data collection approach was cited as a major challenge.135 For the past four to five months in Ethiopia, and for close to a year in Kenya, South Sudan, and Uganda, data collection by field monitors has stopped (due to termination of their contracts).136 A second challenge is the nature of the cross-border conflicts. Conflicts are cyclical where pastoralist communities seek scarce water and pasture in arid areas affected by variable climate, conflicts are cyclical.137 Given this cyclical nature, one might argue that these conflicts are predictable and, therefore, more easy to prevent and/or manage. However, these areas are characterized by a minimal and, in some cases, absent state presence. Additionally, uncoordinated state responses have left stabilized communities vulnerable to their cross-border neighbors. For example, Uganda, pursued a sustained disarmament and development program, while South Sudan did not, and Kenya only temporarily implemented it. 138 Finally, resource constraints, either due to a lack of funds or slow disbursements have hampered completion and expansion of peace dividend projects.139 Conclusion Because the rapid response to cross-border conflicts—or to any conflicts for that matter—is dependent on timely alerts, the lack of a data collection system hinders the usefulness of the CEWARN mechanism. Additionally, concerning cyclical, cross-border conflicts: measuring progress is a challenge since achievements can be time-specific and not permanent. Where local capacities for peace and security fail, there is a vicious cycle of conflict. This further emphasizes the importance of long-term peacebuilding, through interventions such as peace dividend projects. Recommendation The CEWARN Secretariat should revitalize its early warning function by reengaging FMs and urgently implementing changes to financial accountability systems to ensure external donor funds are correctly used in a timely manner. Recommendations five, six and seven in the Recommendation Section 5.0 directly or indirectly apply to this question. Evaluation Question 2c: To what extent can CEWARN be expected to effectively contribute to advancing peace and stability in the East Africa region it serves? As discussed above, CEWARN Secretariat systems are currently on hold and only remnants of the 135 KIIs, member state agency staff, CSO/NGO staff and community members in Ethiopia, Kenya, South Sudan, and Uganda, August - October 2015 136 Ibid. 137 KIIs, member state agency staff, CSO/NGO staff, and community members, Ethiopia and South Sudan, September 2015 138 KIIs, member state agency staff, community members Uganda, September 2015 139 KIIs, member state agency staff, CSO/NGO staff, Ethiopia, September 2015 30 pre-mid 2014 system are currently operating. 140 The CEWARN Secretariat also appears to lack a core business planning approach that focuses on appropriate regional service provision. It is, therefore, failing to effectively advance or identify important opportunities to improve its role. 141 The CEWARN Secretariat’s ability to satisfy external donor liquidation or accounting/auditing requirements in relation to funds provided will also hamper any progress. Funds will continue to be withheld pending satisfactory compliance with these financial requirements. 142 Conclusion Addressing the above issues will be critical to future progress. The evaluation team strongly believes that such issues must be addressed as an urgent priority by all stakeholders. This is because evidence gained from the system’s operation prior to mid-2014 showed that regionally standardized conflict early warning and response mechanisms can function effectively and produce significant benefits for all member states. Recommendation Please see Annex 12 for recommendations on how the CEWARN system might be re-established. Cross-cutting recommendations five through seven and ten in Recommendation Section 5.0 directly or indirectly apply to this question. 4.3.3 CEWARN Evaluation Question 3 To what extent do civil society organizations in the East Africa region work with and support CEWARN and/or the CEWERUs? What recommendations do the regional stakeholders, including CSOs, have for improving the effectiveness of the CEWARN and CEWARU programs? Evaluation Question 3a: To what extent do civil society organizations in the East Africa region work with and support CEWARN and/or the CEWERUs? Because the CEWARN Secretariat’s conflict and early warning system is currently on hold, there is little formal connection between the CEWARN Secretariat and CSOs. However, under the system prior to mid-2014 there were strong links between the CEWARN Secretariat, CEWERUs, and the CSOs. Many of the member states’ conflict early warning systems were actually designed and initiated by the CSO/NGO community and later adopted by the CEWARN Secretariat and many of the ex-FMs were employees of the CSOs. In most member states, the CSOs were largely responsible for testing the system and in some instances the CSOs were involved in the development of CEWARN’s 2012 to 2019 Strategy Framework. 143 Both the CEWARN Secretariat and the external donors appear to have overlooked many CSO innovations adopted under the pre￾existing system when designing the new system. The new system proposes changes to the substantial, pre-existing hard won CSO roles, including immediate participation in initiating locally￾designed responses. Such proposals have alienated many within this particular group of stakeholders.144 To the extent that CEWERUs are still operating the remnants of the pre-existing system, the strong links between CEWERUs and CSOs remain. For example, the Kenyan CEWERU still maintains a large number of FMs embedded in CSOs and these CSOs provide a very strong advisory capacity to the CEWERU.145 140 KIIs, member state agencies, Ethiopia, Kenya, South Sudan, and Uganda, August - September 2015 Early warning in the Horn of Africa, Press Release, SaferWorld, August 2015. 141 KIIs, member state agencies, Ethiopia, Kenya, South Sudan, and Uganda, August - September 2015 CEWARN 20012 to 2019 Strategy Framework, p. 21 142 KIIs, USAID staff, Kenya, October 2015; IGAD Organizational Structure Review, GiZ, June 2006. pp. 12-16 143 KIIs with CSOs/NGOs in South Sudan, Ethiopia, Uganda, and Kenya, August - September 2015 144 Ibid. 145 KIIs, member state agency, Kenya and Kenyan CSOs/NGOs October 2015 31 Evaluation Question 3b: What recommendations do the regional stakeholders, including CSOs, have for improving the effectiveness of the CEWARN and CEWARU programs? The most common request from government agencies and CSOs consulted by the evaluation team was for CERWARN to abandon the proposed new regional conflict early warning and response system and to focus on consolidating and improving the previous system. Suggestions from CSOs included: revising approaches in the management of the RRF so it can be legitimately used as a rapid response resource; undertaking clearer planning that identifies the CEWARN Secretariat’s key roles and responsibilities as a regional agency; and increasing the CEWARN Secretariat’s opportunities to play a unique or advantageous role.146 The latter suggestions are related to opportunities for the CEWARN Secretariat to assist CSOs that are currently neglected or overlooked. Such opportunities include roles in developing codes of conduct for all players in conflict responses and in improving research and good practice dissemination for the region. Recommendation In recognition that the new system had good intentions, it is urgent that a compromise is developed. The new system must incorporate the lessons learned from the previous system, but can also identify means of expanding CSO involvement in ways that are consistent with the lessons learned. See recommendations five through seven in Recommendation Section 5.0 which directly or indirectly apply to this question. 4.4 IDDRSI The IDDRSI strategy is aimed at addressing the effects of drought and related shocks in the IGAD region in a sustainable and holistic manner. The decision to end drought emergencies was taken by IGAD and the East African Community (EAC) Heads of State and Government at a summit convened in Nairobi, September 9, 2011, following the severe drought that devastated the region in 2010 and 2011. The summit participants took the bold decision to address the effects of recurring droughts on vulnerable communities in the IGAD region and called for an increased commitment by drought-affected countries and development partners to support investments in sustainable development in the Arid and Semiarid Lands. The summit assigned the IGAD Secretariat with the role of leading and coordinating the implementation of the decision and urged all countries and all concerned to work together as a region. Summit participants urged stakeholders to strive to do things differently, work concertedly and holistically, and combine relief and development interventions to build resilience against future shocks. The preparation of the IDDRSI strategy was an inclusive and participatory process that involved the staff of the IGAD Secretariat and IGAD specialized institutions as well as public and non-state actors in member states. The strategy was further informed by consultations with other stakeholders commonly affected by drought or involved in responding to its effects, including CGIAR and UN agencies and development partners. The development of the strategy was guided by the IGAD 2011 to 2015 Strategy and was defined in scope, rationale, and justification by the region’s SWOT and PESTLE analyses. The strategy defines its vision, mission, and overall goal, envisioning a region with communities free from vulnerabilities to drought emergencies. The strategy proposes operational and institutional implementation plans and a results-based M&E system to track the progress of project activities. The IDDRSI mission is to enhance drought disaster resilience and sustainability in the IGAD region. Its strategy recognises the need for a comprehensive and holistic approach to combating chronic food and nutrition insecurity and addresses issues of deep-seated poverty and environmental degradation in order to build the resilience of communities and households against the effects of droughts and other shocks. The strategy identifies seven priority interventions where the necessary 146 KIIs with CSOs/NGOs Uganda, Ethiopia, South Sudan and Kenya, August/September 2015 32 investment and action will help build resilience by reducing the vulnerability of target communities to climatic and economic shocks. These priority interventions include: 1) ensuring equitable access and sustainable use of natural resources, while improving environmental management; 2) enhancing market access, facilitating trade, and offering versatile financial services; 3) providing equitable access to livelihood support and basic social services; 4) improving disaster risk management capabilities and preparedness for effective response; 5) enhancing the generation and use of research, knowledge, technology, and innovations in the IGAD region; 6) promoting conflict prevention and resolution and peace building; and 7) strengthening coordination mechanisms and institutional arrangements for more organised, collaborative, and synergistic action, as well as improving partnerships to increase the commitment and support necessary to execute the objectives of the initiative. The strategy served as a common framework for developing national and regional programs designed to enhance drought resilience with sustainable interventions in the IGAD region. The strategy will guide and inform the process of implementing the drought resilience initiatives at the national, regional, and international levels and will be united and harmonised under the overall coordination and leadership of the IGAD Secretariat. The strategy, by design, recognises that while drought-prone communities face common challenges and are often interconnected through shared natural resources, regional trade, and trans-boundary human and animal movements, individual IGAD member states may have their own specificities and areas of emphasis. IDDRSI Achievements to Date IGAD capitalized on the international, regional, and national political commitment for a regional approach to addressing disaster resilience in the Greater Horn of Africa. Besides establishing a drought resilience platform, member states also agreed to a common programming framework captured in both the regional and country programming papers.147 A Resilience Analysis Unit was set up to build the capacity of IGAD’s primary stakeholders in the measurement and analysis of resilience among vulnerable households and communities.148 IGAD also recruited IDDRSI national coordinators,149 and with USAID funding, hired 11 finance, administration, and technical staff at IGAD’s secretariat, including within the IDDRSI PCU. 150 Overall, IGAD has mobilized over $600 million from development partners for disaster resilience projects in the region.151 The IDDRSI strategy framework was developed with the following seven priority intervention areas: 1) natural resources and environment management; 2) market access, trade, and financial services; 3) livelihood support and basic social services; 4) disaster risk management, preparedness, and effective response; 5) research, knowledge management, and technology transfer; 6) conflict prevention, resolution, and peacebuilding; and 7) coordination, institutional strengthening, and partnerships. As of 2013, USAID, through its Joint Planning Cell Activities and Programs, has made the following contributions to IDDRSI’s seven priority intervention areas (listed above). 147 IDDRSI. 2013. Report of the 3rd Meeting of the Interim Platform Steering Committee, p.2 148 IGAD. Resilience Analysis Unit: Measuring, Understanding and Building Resilience of Vulnerable Populations in the Horn of Africa. p.2 (document not dated) 149 IGAD. 2015. IDDRSI Programming Report: 4th IDDRSI Platform Steering Committee Meeting. Venue: Addis Ababa, Ethiopia, 25 – 26 March, p.21 150 KII with IGAD Secretariat staff, Djibouti, August 2015 151 IGAD. 2015. IDDRSI Programming Report: 4th IDDRSI Platform Steering Committee Meeting. Venue: Addis Ababa, Ethiopia, 25 – 26 March, p.6 33 Table 3: IDDRSI’s contributions IDDRSI Priority Intervention Areas Joint Planning Cell Activities and Programs i ii iii iv v vi vii ETHIOPIA The Pastoralists Areas Resiliency Improvement through Market Expansion (PRIME) Program ✔ ✔ ✔ ✔ ✔ ✔ Livelihoods Integration Unit (LIU) in the Ministry of Agriculture ✔ ✔ ✔ Land Administration and Nurture Development Program (LAND) ✔ ✔ ✔ Water, Sanitation and Hygiene Transformation for Enhanced Resilience Program (WATER ) ✔ ✔ Building Resilience Project in Gode Zone, Somali Region ✔ ✔ ✔ Revitalizing Agricultural/Pastoral Incomes and New Markets (RAIN) program extension known as RAIN + ✔ ✔ ✔ ✔ Productive Safety Net Program (PSNP) ✔ ✔ ✔ Governance and Conflict Prevention (SIPED2) designed but not contracted ✔ ✔ Work Force Development designed but not finalized ✔ Knowledge Learning and Dissemination Program (KLDP) ✔ ✔ ✔ ✔ ✔ ✔ ✔ KENYA Resilience and Economic Growth in the Arid Lands– Improving Resilience (REGAL-IR) ✔ ✔ ✔ ✔ ✔ ✔ ✔ Resilience and Economic Growth in the Arid Lands– Accelerated Growth (REGAL-AG) ✔ ✔ World Food Program Cash and Food for Assets Program ✔ ✔ ✔ APHIAPlus Integrated Marginal Arid Regions Innovative Socialized Health Approach ✔ ✔ APHIAPlus Integrated Marginal Arid Regions Innovative Socialized Health Approach ✔ ✔ ✔ ✔ Northern Rangelands Trust ✔ ✔ ✔ ✔ ✔ EAST AFRICA REGION IGAD Secretariat Support ✔ ✔ Technical Consortium for Building Resilience to Drought in the Horn of Africa ✔ ✔ Animal Health Control across the IGAD Region ✔ ✔ ✔ ✔ Regional Resilience Knowledge Management and Learning (planned awaiting contracting) ✔ PEACE III program (planned awaiting contracting) ✔ Source: USAID. 2013. Horn of Africa Joint Planning Cell Annual Report. P.12 4.4.1 IDDRSI Evaluation Question 1 How has IGAD leveraged USAID support to strengthen the PCU to implement the IDDRSI strategy? Administrative support positions have facilitated all IGAD functions across the IGAD Secretariat, not just those relating to the IDDRSI platforms. Technical positions have been largely focused on specific IDDRSI projects, but have also contributed to the development and promotion of IDDRSI CPPs, within IGAD, among member states, and externally.152 One of the PCU’s key functions is to promote coordination among all IGAD programs. In addition to promoting the coordinating roles of the CPPs, USAID-supported staff assisted in the convening of regular cross-program coordination forums.153 Enhanced capacity in the PCU has not, however, resulted in directly leveraging internal 152 KIIs IGAD Secretariat, Djibouti; USAID Kenya August - October 2015 153 KII IGAD Secretariat, Djibouti, August 2015 34 IGAD cooperation. This may be due to the often limited ownership of the CPPs by other IGAD agencies and the PCU’s unwillingness to coordinate in other ways such as joint planning.154 According to the IGAD Secretariat, IGAD will not be able to absorb the USAID-funded positions as permanent positions by the end of the current round of funding.155 At this point in time, no steps are being made to incorporate any of the positions supported by USAID as permanent IGAD staff positions.156 Absorption as permanent staff is virtually impossible to achieve without revising the official staffing structure of the Secretariat. This would require substantial consultation of and eventual ratification by member states because the salaries of all IGAD’s permanent staff are required to be paid from member state contributions. While IGAD respondents noted that revision of the official staffing structure has been discussed internally, they concurred with the Secretariat that the time remaining in the current funding cycle was insufficient to accomplish this. IGAD respondents noted that external funding would be necessary to support the revision and, therefore, addressing this challenge will take time.157 On a broad scale, USAID support for the IDDRSI strategy has created international visibility for IGAD’s regional role.158 This, in turn, has resulted in more donor interest in the IDDRSI platform, which has supported IGAD’s fundraising activities.159 Conclusion IGAD leveraged USAID assistance in strengthening the PCUs and was able to secure funding from other donors as a result.160 That said, all expansion efforts based on the capacity represented by these positions are in danger of regressing due to the Secretariat’s inability to absorb the positions funded by USAID. Recommendation To prevent the loss of staff and thus capacity in the future, donors may wish to consider other models of assisting IGAD agencies with capacity building. If a needed external capacity exists in the agency’s local area, assistance may be given to engage such external service providers. If local external capacity does not exist, persons recruited to fill such gaps could be groomed to provide ongoing consultancy services once donor funding ends. 4.4.2 IDDRSI Evaluation Question 2 Since its inception, how has USAID support developed the foundations for knowledge management in IGAD? What challenges have been experienced by IGAD in operationalizing the knowledge management system? How have these challenges been handled? Evaluation Question 2a: Since its inception, how has USAID support developed the foundations for knowledge management in IGAD? Measuring USAID’s contribution in developing IGAD’s institutional capacity on knowledge management is challenging. USAID provided support to recruit 11 staff to work in the various IGAD units. The knowledge management coordinator position was initially filled, but the staff member left in mid-2014. Re-recruitment was delayed because the IGAD recruitment processes did not attract a candidate that was acceptable to USAID. Revision of the recruitment processes to capture a wider 154 KIIs USAID and multilateral agencies Kenya, September - October 2015 155 KII IGAD Secretariat, Djibouti, August 2015 156 Ibid. 157 Ibid. 158 Ibid. 159 Ibid. 160 KIIs with USAID staff September - October 2015 35 pool of candidates has been proposed but not yet implemented.161 The knowledge management coordinator enabled the development of knowledge management products, such as the installation of a server in Ethiopia. IGAD was also able to coordinate knowledge management activities within its institutions.162 While the respondents did not dispute that IGAD may have an important regional role to play in knowledge management, they expressed concern that a lack of clear definition and delineation of this role could lead to duplicative and competing efforts.163 The lack of such clarity is evident in the confusion expressed by respondents in relation to the intent of IGAD adopting a knowledge management role.164 Suggestions by respondents for appropriate roles of IGAD knowledge management included:165  M&E of all its activities, including those of its agencies (particularly in regard to defining indicators for and measuring progress against strategies or plans);  M&E of coordination efforts between all IGAD activities;  Storing of metadata regarding information available on key regional issues and providing the links, contacts, and introductions required to facilitate access to such data by both IGAD agencies and others working for the betterment of the region; and  Holding forums and advocating for: o Regional standardization of data collection to improve potential for aggregation of datasets, and o Improving access to data for member state agencies and any other organizations holding data of use to IGAD agencies or other stakeholders. In terms of resilience issues, the third main bullet point (storing of metadata) would not prevent IGAD from storing selected raw data,166 from considering it in its internal monitoring system, or storing a standardized set of regional resilience indicators for its own analyses. However, given its (apposite) heavy reliance on external organizations for most of its on-the-ground delivery of resilience programs, it may not be appropriate for IGAD to try to define such indicators independently. This would be a wasteful duplication of the existing technical capacity of other implementing agencies operating in the region.167 Facilitation of multi-stakeholder agreements on a regionally standardized set of indicators would be a more appropriate role for IGAD (see fourth bullet point above) and one that it is uniquely placed to deliver.168 Conclusion USAID assistance in this area has primarily been in the form of increased human capacity during the period the knowledge coordinator position was filled. However, it should be noted that the quality of IGAD knowledge management is not just dependent on personnel availability. How knowledge is managed is at least equally important. Any attempts by IGAD knowledge management initiatives to depend primarily on storage of externally generated raw data is illogical because each raw dataset would require its own set of fixed data-table structures. Constructing such structures for all useful and available information in the region would be prohibitively difficult. Limiting data structures to manageable complexity would necessarily mean excluding a great deal of available information. Trying to maintain the currency of a wide variety of externally generated data would be equally prohibitive and impractical. These findings apply not just to knowledge management within the PCU of IDDRSI. The CEWARN Secretariat will need to review its data storage and analysis systems in light of these considerations. ICPAC is already reasonably selective about the raw data it stores and relates this to product development, however, forward-looking core business planning remains 161 KII IGAD Secretariat, Djibouti August, 2015 162 Ibid. 163 KIIs with USAID staff September - October 2015 164 Ibid. 165 KIIs with USAID/Kenya and USAID/EA E staff; multilateral agencies, Kenya September 2015 166 Including data layers for GIS systems. 167 KIIs with IGAD Secretariat, Djibouti, USAID/Kenya, and USAID/EA staff, August - September 2015 168 KII USAID/Kenya USAID/EA staff, September 2015 36 necessary to define future development of knowledge management systems. Recommendation The IGAD Secretariat (incorporating the IDDRSI PCU) should focus on storing only carefully selected raw data, such as that generated internally for M&E needs or required to generate key products and services identified in core business planning. It should also design a complementary data structure that references external data sources, as this is a more efficient and achievable option that prevents much of the workload associated with updating externally sourced datasets. Evaluation Question 2b: What challenges have been experienced by IGAD in operationalizing the knowledge management system? How have these challenges been handled? A number of stakeholders expressed concerns with the independent direction taken by IDDRSI in relation to knowledge management. According to these respondents, the approach seems to duplicate the roles of existing organizations in the region or overlook important new roles it is uniquely positioned to fill.169 These concerns were backed by clear and valid examples, such as attempts to duplicate existing datasets and a lack of advocacy for improved regional data sharing.170 The team could find no evidence that defining a clear role for knowledge management within IGAD and harmonization of roles across IGAD agencies have taken place. In general, IGAD has been addressing the challenge of high staff turnover. More specific challenges are related to recruitment and to retaining a knowledge management coordinator within the PCU. The recruitment process was initially stalled due to the delay in the release of funds by USAID and mixed signals on whether or not support would continue.171 While a satisfactory candidate was initially identified and recruited, s/he remained in the position for only a year before leaving for a position with another organization.172 Recruitment was initiated immediately to fill the position and a candidate was selected and presented to USAID for approval. However, the candidate did not meet USAID requirements.173 To address the challenge of finding a suitable knowledge management coordinator, and recognizing that the vacancy seriously impeded progress, in August 2015, IGAD took the step of appointing an existing staff member to act in this position. IGAD also committed to fast-tracking existing recruitment processes to permanently fill the position and to extend the range of media locations for the advertised position. IGAD hopes that knowledge management will be a priority area for future funding from USAID or other interested donors.174 Conclusion The implementation of a robust knowledge management system has been hampered by the lack of defined roles and responsibilities and the difficulty in recruiting and retaining a coordinator. There are many inherent constraints in recruiting high-quality staff for work in Djibouti. The climate, general lack of local facilities, relatively high cost of living, and isolation all work against individuals accepting positions there. IGAD salary structures are also considerably lower than agencies that compete for similar staff. Recommendation As a starting point, the team recommends defining the roles and responsibilities of IGAD and its agencies in the coordination of a robust knowledge management system. To assist in this discussion, the team suggests the following could be used as guidance:  Clearly identify the knowledge management initiatives’ priority clients in the region;175  Clearly identify what knowledge management products and services are most needed (and, therefore, most highly valued) by high-priority, regional clients; 169 KIIs USAID/Kenya USAID/EA staff and multilateral stakeholders, August/September 2015 170 Ibid. 171 Group discussion with IGAD, Djibouti, August 2015 172 KII IGAD Secretariat, Djibouti August, 2015 173 KII, IGAD Secretariat, Djibouti; USAID/Kenya and USAID/EA staff, August/September 2015 174 KII, IGAD Secretariat, Djibouti, August 2015 175 Note that external donors are not regarded as direct clients, but rather as supporters of service provision to regional clients (this keeps the system outcomes focused, as is of most benefit to external donors). 37  Clearly identify the key knowledge management products and services that are already satisfactorily provided by others in the region;  Clearly identify which of the remaining key knowledge management products and services align with IGAD’s technical and political capabilities; and  Clearly identify constraints that may limit IGAD’s mandated roles or powers in relation to knowledge management. In addition, the team recommends that IGAD reviews what attributes, experience, and education will be needed for the position of coordinator, determine a realistic package of benefits needed to recruit and retain an incumbent, and negotiate with USAID for the necessary funding. Also see recommendations two, four, and eight in Recommendation Section 5.0, which directly or indirectly apply to this question. 4.4.3 IDDRSI Evaluation Question 3 USAID/EA has supported the institutional capacity strengthening of IGAD by providing resources for IGAD to hire core finance and administration staff. To what extent have the expected results and/or outcomes been achieved? What steps is IGAD taking to assume responsibility for funding of these staff to avoid dependence on USAID? What is the anticipated timeline for IGAD to fully assume funding responsibility for these staff? Evaluation Question 3a: USAID/EA has supported the institutional capacity strengthening of IGAD by providing resources for IGAD to hire core finance and administration staff. To what extent have the expected results and/or outcomes been achieved? The IGAD staffing structure is ten years old, and its revision has been ongoing (based on Ernst and Young’s 2004 institutional assessment report recommendations).176 Only when the revision is complete and the new positions have been created can these new hires become regular employees. USAID/EA’s institutional capacity strengthening support to IGAD’s finance and administration staff was for eight staff positions—seven in the area of procurement and financial management, and one as a technical assistant to the executive secretary.177 Expected results or outcomes of this investment are implied and not defined conclusively in the LSGA; they were never actually defined.178 While a conclusive set of outcomes was never followed-up on, the implied results were interpreted by IGAD to show an improvement in the IGAD Secretariat’s finance and administration capacity. IGAD hired all eight staff in line with the agreement. Based on discussions with the finance and administration team, this staff’s capacity to efficiently manage procurement, human resource management, and financial management and reporting has improved, but not optimally.179 They did confirm the need for support for more staff.180 The procurement unit is new and the feeling by the finance and administration team was that the unit is not fully understood yet within IGAD. The unit is thinly staffed and needs to be further strengthened to perform efficiently.181 Staff turnover is high due to IGAD’s uncompetitive terms of service relative to the African Union or the East African Community.182 176 Group interview, IGAD Secretariat, Djibouti, August 2015 177 USAID/EA, Limited Scope Grant Agreement (LSGA no. 623-LSGA6230009.02-3-60082): Amendment Number Seven between the United States of America acting through the United States Agency for International Development (USAID) and The Inter￾Governmental Authority on Development (IGAD), September 2012, Annex 3, p.5 178 Ibid, p.8. There may be later amendments to the LSGA, but the evaluation team was provided only Amendment number seven. 179 Group interview, IGAD Secretariat, Djibouti, August 2015 180 Ibid. 181 Ibid. 182 Group interview, IGAD Secretariat, Djibouti, August 2015 38 Conclusion The sustainability of the gains made in capacity strengthening is unclear barring continued donor support; it is uncertain if IGAD will be able to take on new staff without USAID funding. Recommendation Recommendations three, four, and eight in Recommendation Section 5.0 directly or indirectly apply to this question. Evaluation Question 3b: What steps is IGAD taking to assume responsibility for funding of these staff to avoid dependence on USAID? As noted above, at this point in time no steps are being made to incorporate any of the positions supported by USAID as permanent IGAD staff positions. Absorption as permanent staff is virtually impossible without revising the official staffing structure of the Secretariat, which would require consultation with and eventual ratification by of member states. This is because the salaries of all the permanent staff of IGAD are required to be paid from member state contributions.183 While IGAD respondents noted that revision of the official staffing structure has been introduced internally, the time needed to achieve this change is greater than the time remaining that remained in the current round of USAID support. Conclusion It is unlikely that IGAD, at least in the short- to medium-term, will be able to function without the support of external donors. It may be possible for IGAD to identify alternate external (project￾based) funding to continue these positions, but this would offer no more guarantee of sustainable capacity development. Recommendation Recommendations three and four in Recommendation Section 5.0 directly or indirectly apply to this question. Evaluation Question 3c: What is the anticipated timeline for IGAD to fully assume funding responsibility for these staff? Based on findings from discussions with IGAD, noted above, IGAD has not proposed a timeline for assuming financial responsibility for these positions.184 Recommendation Recommendations two, three, four, and eight in Recommendation Section 5.0 directly or indirectly apply to this question. 4.5 ICPAC185 The Greater Horn of Africa is prone to extreme climate events such as droughts and floods. These extreme events have severe negative socio-economic impact in all the countries in the sub-region. In 1989, 24 countries in Eastern and Southern Africa established a Drought Monitoring Center (DMCN) headquartered in Nairobi with a sub-center in Harare (Drought Monitoring Centre Harare, DMCH) to respond to devastating weather-related disasters. In October 2003, the Heads of State and Governments of IGAD held their 10th Summit in Kampala, Uganda, where the DMCN was adopted as a specialized IGAD institution. The name of the institution was changed to IGAD Climate Prediction and Applications Centre (ICPAC) to better reflect its mandates, mission, and objectives within the IGAD system. A protocol fully integrating the institution into IGAD was signed on April 13, 2007. The center is responsible for seven member countries, Djibouti, Eritrea, Ethiopia, Kenya, 183 Article 10., AGREEMENT ESTABLISHING THE INTERGOVERNMENTAL AUTHORITY ON DEVELOPMENT (IGAD) 184 KII IGAD Secretariat staff, Djibouti, August 2015 185 http://rcc.icpac.net/contents.php?sm_id=1&tm_id=1&cdepth=2&upnum=2&ca_id=3&s1=2&t1=2 39 Somalia, South Sudan, and Uganda, and three non-member countries, Burundi, Rwanda, and Tanzania. ICPAC’s vision is to become a viable regional center of excellence in climate prediction and development of applications for climate-risk management, environmental management, and sustainable development. The objectives of the center are: 1. To provide timely climate early warning information and support specific sector applications for the mitigation of the impacts of climate variability and change for poverty alleviation, environmental management, and sustainable development; 2. To improve the technical capacity of producers and users of climatic information, in order to enhance the use of climate monitoring and forecasting products in climate risk management and environment management; 3. To develop an improved, proactive, timely, broad-based system of information/product dissemination and feedback, at both sub-regional and national scales through national partners; 4. To expand climate knowledge base and applications within the sub-region in order to facilitate informed decision making on climate risk related issues; and 5. To maintain quality controlled databases and information systems required for risk/vulnerability assessment, mapping, and general support to the national/regional climate risk reduction strategies. 4.5.1 ICPAC Evaluation Question 1 To what extent has ICPAC been effective and efficient in achieving its strategic objectives and responding to climate change priorities? Table 4: ICPAC findings against objectives ICPAC Objective Findings Provide timely climate early warning information and support specific sector applications for the mitigation of the impacts of climate variability and change for poverty alleviation, environmental management, and sustainable development  ICPAC has reliably delivered forecasting products and services to member states Improve the technical capacity of producers and users of climatic information, in order to enhance the use of climate monitoring and forecasting products in climate risk management and environment management  ICPAC has trained staff in national meteorological agencies  ICPAC has provided IT equipment to meteorological agencies  ICPAC is involved in data rescue of paper-based climatic records  ICPAC is standardizing climate modeling/prediction applications e.g., GeoCLIM Develop an improved, proactive, timely, broad￾based system of information/product dissemination and feedback, at both sub￾regional and national scales through national partners  ICPAC Climate Outlook Forums have been valuable for joint technical discussions  ICPAC is successful in broader level communication of its forecasts to governments and other stakeholders 40 Expand climate knowledge base and applications within the sub-region in order to facilitate informed decision making on climate risk related issues  ICPAC Climate Outlook Forums have been valuable for joint technical discussions  ICPAC is successful in broad communication of its forecasts to governments and other stakeholders  ICPAC is standardizing climate modeling/prediction applications e.g., GeoCLIM Maintain quality controlled databases and information systems required for risk/vulnerability assessment, mapping and general support to the national/regional climate risk reduction strategies  ICPAC has reliably delivered forecasting products and services to member states  ICPAC is standardizing climate modeling/prediction applications e.g., GeoCLIM At the most basic level, meeting of all ICPAC’s objectives hinges on the provision of regional and seasonal forecasting products and services to member states. It has been able to reliably deliver these products and services to member state meteorological agencies and other clients.186 Respondents from national meteorological agencies of IGAD member states unanimously voiced strong appreciation of these products and supported their position with examples of direct use, particularly in relation to providing critical advice to national agricultural agencies and farmers.187 ICPAC also provides critical capacity building support to national meteorological agencies, including provision of IT equipment and training. While staff turnover in these national agencies sometimes defeats capacity-building efforts, and ICPAC capacity building effects have not been monitored or evaluated formally, most respondents agreed that a steady, if relatively slow, improvement in member state meteorological agency capacity is attributable to ICPAC.188 Other forms of important capacity building raised by respondents were ICPAC’s ongoing involvement in “data rescue” of paper-based climatic records and its role in promoting standardization of climate modeling/prediction applications, such as GeoCLIM.189 The Climate Outlook Forums (COFs) facilitated and conducted by ICPAC also are reported as being strongly supported and valuable.190 Attendance at the most recent COFs averaged 100 to 200 participants, including attendees from many external organizations. This is well in excess of the default attendance of approximately five or so officers from each of the member state meteorological agencies.191 The venue for the COFs rotates through IGAD member states. The hosting member state agency provides significant logistical assistance.192 These COFs are reported to be invaluable for enabling technical discussions regarding the interpretation and implications of climate data at both expert and practitioner levels.193 ICPAC has been particularly successful in broadly communicating its forecasts and the forecasts’ importance. For example, ICPAC has been the key player in communicating the likely effects of the current El-Nino to governments and other stakeholders.194 The seriousness with which member states now take ICPAC forecasting, including climate unpredictability and change, is reflected in recent public policy developments and action planning conducted in member states most affected by 186 KIIs, member state agencies, Djibouti, Kenya and Uganda August/September 2015 187 Ibid., ICPAC Organizational Self-Assessment, ICPAC, 2010 p. 13-15 188 KIIs, member state agencies, Djibouti, Kenya, and Uganda; KIIs with ICPAC and USAID staff, Kenya August - October 2015 189 KII with staff of USAID activity, Kenya, October 2015 190 KIIs with member state agencies, Djibouti, Kenya, and Uganda and KII with USAID staff August, October 2015 191 KII ICPAC, Kenya, October 2015; COF 39 Bulletin, ICPAC, 2015, pp. 1-2 192 KII ICPAC, Kenya, October 2015; KII member state agency, Djibouti, August 2015; COF 39 Bulletin, ICPAC, 2015, pp. 1-2 193 KIIs with member state agencies, Djibouti, Kenya, and Uganda; KII with USAID staff Kenya, August – October 2015; COF 39 Bulletin, ICPAC, 2015, pp. 1-2 194 KIIs USAID and multilateral agencies, Kenya October 2015; KII with member state agency, Uganda September 2015 41 the current El-Nino.195 Evaluation respondents from the national meteorological agencies credited ICPAC with the acceptance and mainstreaming of climate variability and change considerations into government decision-making and resource allocation. ICPAC, with assistance from media outlets, has been successful in generating awareness of such climate issues and their implications well beyond technical and administrative circles.196 Impromptu discussions initiated in member states visited by evaluation team members with “unconnected” members of the public revealed an almost-universal public awareness of the current El-Nino occurrence and its likely implications. ICPAC faces the same human resource constraints found within IGAD and its agencies and has very few permanent technical staff.197 Additionally, many stakeholders pressure IPAC to stray from its primary roles of seasonal and regional forecasting.198 Conclusion ICPAC appears to have done well in meeting its main objectives. This may be attributable to its strategy which clearly identifies its role, down to the level of key products and services. 4.5.2 ICPAC Evaluation Question 2 To what extent are USAID-supported programs responding to thematic priorities as outlined in ICPAC’s strategic plans and other regional strategies agreed to by member states? Linked initiatives such as the USAID-funded PREPARED activity and the Famine Early Warning System Network (FEWS NET) are generally working in concert with the ICPAC and IDDRSI strategies.199 For both PREPARED and FEWS NET, the strongest interaction is overwhelmingly with ICPAC.200 PREPARED also has assisted ICPAC by providing climate modelling software and training, making ICPAC a leader in certain capabilities, such as downscaling of climate predictions to specific areas.201 This ability is in high demand by member states.202 Interaction with IDDRSI has largely been limited to member state intermediaries, which is not surprising given IDDRSI’s coordinating role.203 This difference interaction is understandable because ICPAC and the head offices of these initiatives are all located in Nairobi. By following the CPPs, USAID activities still provide contributions to the IDDRSI strategy. Another factor that argues against USAID deviation from the IDDRSI strategy is that it is encompasses a wide range of resilience issues and programming; it is likely that any resilience-related activities will be nominally in line with the IDDRSI strategy.204 195 KII ICPAC staff, Kenya, September 2015; Ministry of Environment, Natural Resources and Regional Authorities, Kenya, Press Release: Short Rains Season, October – December 2015; Region told to prepare for severe El Nino rains, The East African, September, 13, 2015 196 KIIs with member states Djibouti, Kenya, and Uganda; KII with USAID staff August – October 2015 197 KII, member state agency Kenya; KII ICPAC Kenya October 2015 198 KIIs, ICPAC, Kenya, October 2015; USAID Kenya, October 2015; ICPAC Organizational Self-Assessment, ICPAC, 2010 p. 6 199 KIIs, PREPARED activity staff Kenya, September 2015; FEWS NET activity staff Kenya, October 2015 Planning for Resilience in East Africa through Policy, Adaptation, Research and Economic Development (PREPARED) ANNUAL PROGRAM STATEMENT (APS), 2015 Evidence-based Analysis for a Food-Secure World, FEWS NET Fact Sheet, 2015. 200 KIIs, PREPARED staff, Kenya, September 2015; FEWSNET staff Kenya, October 2015 201 KII PREPARED activity staff, October 2015; 2015 Planning for Resilience in East Africa through Policy, Adaptation, Research and Economic Development (PREPARED) ANNUAL PROGRAM STATEMENT (APS), 2015 202 KII PREPARED activity staff, October 2015 203 KIIs PREPARED and FEWS NET activity staff, Kenya, September/October 2015; The IDDRSI Strategy, IGAD, 2013 pp. 20 - 23 204 KII, USAID Project, Kenya, October 2015; The IDDRSI Strategy, IGAD, 2013 pp. 20 – 23; IDDRSI Programming Report, March 2015; pp. 20-33 42 Risks associated with unpredictable funding availability, which affect all IGAD agencies (see also Evaluative Question 1a), also affect ICPAC due to the significant and increasing demand for new regionally-developed climate products and services and an expectation by member state agencies and donors that ICPAC should be the one to deliver these products.205 Conclusion ICPAC has been effective and efficient in delivering highly-valued products and services to regional clients. It has demonstrated a clear ability to communicate climate variability and change issues to governments and broader stakeholders, especially in relation to the current El-Nino event. Uncoordinated demand from stakeholders for new climate products and applications has the potential to pull ICPAC in many directions, which could be problematic. If PREPARED’s role in conducting core business planning for member state meteorological agencies could be expanded to include ICPAC, resultant core business plans could be used by both ICPAC and external stakeholders in defining the possibilities and priorities for medium-term expansions in product and service provision. The plans also would be useful for informing the future direction of any available external funding. Recommendation USAID should expand PREPARED’s role in conducting business planning for member state meteorological agencies to include ICPAC. Additionally, recommendations two, three, and ten in Recommendation Section 5.0 directly or indirectly apply to this question. 4.5.3 ICPAC Evaluation Question 3 How are current systems and processes of ICPAC helping or hindering it from meeting regional climate change demands and priorities? Risks associated with unpredictable funding availability that affect all IGAD agencies (see also Evaluative Question 1a), also affect ICPAC due to the increasing demand for new regionally￾developed climate products and services and member states’ and donors’ expectations that ICPAC should be the one to deliver these products.206 While existing ICPAC staff are considered to have strong technical capacity, there is a prevailing perception that ICPAC lacks sufficient human resources to cope with existing workloads.207 This view is supported by the routine borrowing of technical staff from the University of Nairobi and the Kenyan Meteorological Department, a practice that has continued over the life of the agency. While this support is welcome and indicative of strong national support for ICPAC, it does not sustainably address the problem of insufficient internal manpower. Conclusion The impact of the unpredictability of funding can be seen in the recurrent problems faced by ICPAC in relation to securing sufficient permanent staff. Recommendation Recommendations two, three, and ten in Recommendation section 5.0 directly or indirectly apply to this question. 205 KII ICPAC, Kenya, October 2015; ICPAC Organizational Self-Assessment, ICPAC, 2010 p. 4 206 Ibid. 207 KIIs, multilateral donor staff, Kenya; member state agencies in Kenya and Uganda, September – October, 2015 43 5 Overall/Cross-cutting Recommendations Recommendation 1: USAID should conditionally renew its assistance to the IGAD secretariat, the IDDRSI PCU, ICPAC, and the CEWARN Secretariat subject to these agencies’ satisfactory compliance with the other recommendations of this report. Recommendation 2: As soon as possible, all IGAD agencies receiving USAID assistance should initiate comprehensive and coordinated core business planning exercises, in line with the approach utilized by the PREPARED project, and including all the following steps:  Clear identification of an agency’s priority clients in the region;208  Clear identification of the products and services that are most needed and, therefore, most highly valued by high-priority clients;  Clear identification of which of these key products and services are satisfactorily provided by others in the region;  Clear identification of the remaining key product and services gaps that align with IGAD agencies’ core capabilities;  Clear identification of real-world constraints that may limit an agency’s mandated roles, powers, or aspirations; and  Specification and monitoring of robust performance and outcome indicators, including specific indicators that measure coordination between agencies. USAID, working individually or with like-minded groups or external donors, should support these exercises. Recommendation 3: All stakeholders must review their understanding of how best to utilize IGAD agencies, especially in relation to correcting misconceptions regarding IGAD’s real-world capabilities and limitations. (A cursory examination of which is provided in Annex 10 of this report.) Recommendation 4: When external donors are considering funding positions within IGAD agencies, they should request specific information on IGAD’s ability to support and maintain the positions in the longer term. IGAD agencies should transparently provide such information. Where necessary, external donors should consider alternate models to boosting IGAD capacity, including the “assistance with outsourcing” or “consultant development” models outlined under Question IDDRSI/PCU 1. Recommendation 5: The CEWARN Secretariat, with assistance from USAID, should urgently redesign the proposed new regional conflict early warning and response system. This revised system must incorporate the lessons learned from the pre-existing system and should consider new means of addressing funding issues, such as the timely liquidation and accounting of expenditures by the retargeting of external donor funds intended to support truly rapid responses at the country level, and subsequent renaming of the CEWARN Secretariat held RRF to the Response Assistance and Research Fund to avoid misleading expectations. It should also consider incorporating a methodology to expand CSO involvement, but only in ways that are consistent with the lessons learned to date. The evaluation team’s assessment of these lessons learned are provided in Annex 11, and detailed direction for associated changes to the currently-proposed system is provided in Annex 12. Recommendation 6: In accordance with recommendations two and five above, the CEWARN Secretariat should add a medium-term boundary to its scope, and consider developing a set of guidelines or a code of conduct for conflict early warning and response for eventual ratification by member states. All agencies likely to be involved in responses to conflicts under any CEWARN￾supported systems would be subject to these guidelines, which should include clear delineation of 208 Note that external donors are not regarded as direct clients, but rather as supporters of service provision to regional clients (this keeps the system outcomes focused, as is of most benefit to external donors). 44 the legitimate roles of CSOs in such responses and discourage potential excesses by armed players. The guidelines should also specifically address expanded measures for ensuring the safety of FMs. Recommendation 7: When providing assistance or advice at a regional level, USAID/Kenya and USAID/EA should remain cognizant of the intent and expected outputs of concurrent bilateral interventions in similar fields to prevent contradictory and inefficient results. For example, support to regional conflict early warning and response mechanisms should not encourage removal of FMs while bilateral projects are in the process of actively training them. Recommendation 8: All IGAD agencies assisted by USAID, but particularly the IDDRSI PCU, should use the core business planning approach specified in recommendation two to guide regional knowledge management approaches, including moving to metadata storage where needed and playing more active roles in the promotion of regional data sharing where feasible. Recommendation 9: External donors, including USAID, should monitor the success and application of the core business planning approaches specified in recommendation two. If satisfactory progress is made in focusing and monitoring IGAD agencies’ activities, then USAID and other external donors should consider providing funds to cover the duration of such core business plans using staged allocations contingent on results from performance/outcome indicators. Recommendation 10: External donors, including USAID, should use all means at their disposal to encourage member states to commit to attending at least one ordinary meeting of the Council of Ministers, annually. Recommendation 11: As a matter of urgency, IGAD should compile and publish the detailed progress made against its 2003 strategy, preferably before or concurrently with adoption of any new strategy. 5 Summary Conclusion Contextual Considerations Regardless of the findings of, or reaction to, this evaluation, IGAD, like many similar organizations around the world finds itself at a crossroad. Some of the reasons this crossroad has appeared are internal—the outcome of formal or informal policies pursued to date. Others are external—money is tight globally, governments and donor agencies are facing much greater scrutiny of expenditures and are adopting approaches to ensure returns on investments in line with the levels more commonly associated with the commercial arena.209 In this new operating environment, impressive pedigrees or mandates are becoming less important, and what now matters is not “who you are,” but “what you can do.” Both formal and informal input to this evaluation by major external donors reveals that they are becoming highly fatigued with organizations that make ambitious claims but demonstrate delivery of vague or less-than-impressive results at best. Donors’ objectives generally focus on improving conditions within recipient countries or regions. So, to consistently attract external funding in such an operating environment, donor-dependent organizations will not only need to focus on service provision, they will have to focus on assured service provision. This will mean being more conservative in claims made in proposals and being far more cognizant of real world constraints, including internal capacities and capabilities. If an organization cannot demonstrate a clear and convincing business case for being both able and likely to produce projected results, it may find external funding increasingly difficult to secure. Should USAID Continue Funding IGAD Agencies? 209 Including ‘payment on delivery’ models. 45 For the responses to the above evaluation questions to be read in the correct context, there is a need to address a key implicit question that underpins all responses: Should USAID funding to IGAD and its agencies continue? While this question is not explicitly stated in the statement of work, its importance was made clear in subsequent meetings between the evaluation team and USAID staff. The team’s position on this question is provided as recommendation one. Key factors that indicate the advisability of further investment include:  IGAD has proven its ability to effectively use its highest level fora (including extensions of these fora to include external stakeholders) to negotiate and aggressively promote solutions to major internal and bilateral conflicts in the region;  Relevant IGAD agencies and platforms have demonstrated the ability to design systems, institutional architecture, and approaches that have been replicated as coordinating standards by member states across the region;210  Many of these coordinated structures and approaches are now regionally-ingrained, and replacing or replicating them independently would be impractical and could potentially damage existing functionality and achievements in member states;211  The many layers of IGAD committees and fora offer immensely valuable opportunities for fostering informal cooperation, communication, and diplomacy, both between and within member states and external stakeholders, which is of critical importance in identifying and implementing durable local solutions to issues in the region, especially regarding conflict and stability;212  IGAD fora and activities have been effectively used to share lessons and promote good practices within the region, which are vital to progressively creating a level playing field of capacity for progress in member states;213  The intergovernmental cooperation facilitated by IGAD initiatives and forums has facilitated significant joint infrastructure and other economic projects that in turn generate regional economic inter-dependence and are consequent drivers of stability;214 and  Under IGAD, practical means of intergovernmental cooperation have generally been rendered operational in this troubled region. While these means may not always take the forms stakeholders prefer, they are likely the only means available. Given that this question was not explicitly asked in the task order, the team was careful not to expend additional resources in attempting to collect data to respond to it. The above points should be regarded as the evaluation team’s opinion, rather than the result of targeted research. 210 KIIs, CSO/NGO; South Sudan, Ethiopia, member state agencies, Uganda and Kenya, September, 2015; Early warning in the Horn of Africa, Press Release, SaferWorld August 2015. 211 Ibid. 212 Ibid. 213 The IDDRSI Strategy, IGAD, 2013 pg. 14; IGAD Organizational Structure Review, GiZ, June 2006. pp. 12-16; ICPAC Organizational Self-Assessment, ICPAC, 2010 p. 13-15 214 KII, IGAD Secretariat; Djibouti, August, 2015; Tripartite & IGAD Infrastructure Investment Conference, EAC Communique, 2012. 46 ANNEXES Annex 1: Map of Participating Countries 47 Annex 2: Statement of Work USAID/EA Evaluation of the Intergovernmental Authority on Development (IGAD) Statement of Work I. Purpose: The purpose of this Statement of Work (SOW) is to seek services from qualified contractors to evaluate the effectiveness and sustainability of USAID/EA-supported activities at the Intergovernmental Authority on Development (IGAD). USAID/EA has provided direct financial support to IGAD since 2006 through a Limited Scope Grant Agreement (LSGA) (623-0009.02). The LSGA duration is from September 24, 2006 through December 31, 2015. USAID/EA’s support to IGAD has funded three IGAD programs:  IGAD Conflict Early Warning and Response Mechanism (CEWARN), $5,269,560 to date;  IGAD Drought Disaster Resilience and Sustainability Initiative (IDDRSI), $1,000,000 to date;  IGAD Climate Prediction and Application Center (ICPAC), $250,000 as of July 2015. II. Background: The recurring and severe droughts and other natural disasters between 1974 and 1984 caused widespread famine, ecological degradation and economic hardship in the Horn of Africa (HOA). Although individual countries made substantial efforts to cope with the situation and received generous support from the international community, the magnitude and extent of the problem argued strongly for a regional approach to supplement national efforts. IGAD was created by member states in 1996, superseding the Intergovernmental Authority on Drought and Development (IGADD) which was founded in 1986. IGAD was established to assist and complement the efforts of member states in the HOA in the areas of food security and environmental protection, promotion and maintenance of peace and security and humanitarian affairs, and economic cooperation and integration. Based in Djibouti, IGAD has seven active member states: Djibouti, Ethiopia, Kenya, Somalia, South Sudan, Sudan, and Uganda. (Eritrea is currently inactive.). Additional information on IGAD can be found at: http://igad.int/. USAID’s Limited Scope Grant Agreement (LSGA) with IGAD has been amended nine times. Initially the agreement supported IGAD’s peace and security objective, implemented by CEWARN. In 2012, the LSGA was broadened to incorporate support for agriculture and environment objectives through (IDDRSI) and (ICPAC). This evaluation of IGAD focuses on the three USAID LSGA-supported activities: CEWARN, IDDRSI, and ICPAC. Each is described below: CEWARN: To institutionalize conflict early warning and response capability, the IGAD Secretariat established the CEWARN Secretariat in 2001. From 2001 to 2004, USAID/EA supported CEWARN through its HOA Support Program. Since 2004, CEWARN activities have been funded directly under an LSGA between USAID/EA and IGAD. IGAD/Conflict Early Warning and Response Mechanism (CEWARN) and national Conflict Early Warning and Response Units (CEWERUs) seek to facilitate local, national, and intergovernmental cooperation and responses to cross-border conflicts in the Horn of Africa. With the IGAD Member States’ adoption of the new 2012-2019 Strategic Framework, CEWARN’s focus expanded dramatically from coordinating responses to pastoral conflict to integrating timely conflict early warning and response analysis into policy making at the local, national, and regional levels. CEWARNs work is transitioning from an exclusive focus on pastoralism to feature new Sectors, including Economic, Environment, Governance, and Security, Social, and Early response. Within 48 these Sectors, thematic areas, including but not limited to election-related violence, land, inter￾ethnic relations, and climate change have been identified. Within this context, it is anticipated that CEWARNs early warning approach will become system-oriented. USAID is supporting CEWARN as to upgrade its systems to fulfill this new mandate as well as to achieve its Strategic Framework Objectives. The CEWARN head office is in Addis Ababa. Its national level units, CEWERU are located in seven IGAD countries: Kenya, Ethiopia, South Sudan, Sudan, Somalia, Uganda, and Djibouti. Each CEWERU brings together government and civil society representatives. For more information please visit: http://www.cewarn.org/. IDDRSI: Seeking to address the catastrophic phenomenon of recurrent droughts and related environmental concerns in a sustainable manner, IGAD heads of state convened a summit in Nairobi in September 2011. The Nairobi Summit made a collective decision to embark on IDDRSI and called for the urgent introduction of strategies, policies and programs that involve the strengthening of investment plans at member state and regional levels with the principal objective of building resilience to future climatic and economic shocks. To drive IDDRSI forward, IGAD member states and their development partners agreed to form a Regional Drought Resilience Platform. The Platform brings together the different partners and stakeholders including member states, the IGAD Secretariat, and development and implementing partners including UN agencies, civil society and specialized research and training institutions. As part of its institutional arrangements, the Platform includes a General Assembly of participating stakeholders, a Platform Steering Committee and a Platform Coordinating Unit. The Platform provides the modalities through which the region’s priorities and possibilities for intervention by affected countries and development partners in support of the drought resilience initiative are discussed – and provides an effective mechanism for coordinating the implementation of the drought resilience initiative. Under the Regional Drought Resilience Platform, a Platform Coordination Unit (PCU) was created and embedded in the IGAD Secretariat. The PCU engages in regular technical and functional contact with IGAD member states and partners to create awareness; plan, execute, monitor and evaluate projects, including development of country programme frameworks and project identification and preparation; provide training and capacity building; prepare reports on programme needs and progress; organize regional technical, policy and coordination meetings; provide technical support as needed; and link activities on the ground with the IGAD Secretariat, Regional Platform and interested partners. The PCU is based in the IGAD Secretariat in Djibouti. The USAID Joint Planning Cell (JPC) for the HOA Resilience Initiative, which is comprised of USAID/EA, USAID/Kenya, USAID/Ethiopia, and the USAID/Washington Bureau for Food Security (BFS), developed a set of integrated regional and bilateral Mission plans, which was approved by USAID Administrator in February 2012. Support for IGAD, funded by BFS and managed by USAID/EA, is a key component of the JPC plan. Additional information can be found at: http://resilience.igadhost.com/index.php/ To support implementation of the IDDRSI Strategy, USAID provides support for two main activities: 1) Build a knowledge management system and 2) Provide institutional strengthening of the IGAD Secretariat through staff support in administration, finance and procurement. ICPAC: ICPAC is a specialized IGAD institution, with eleven member countries, whose mission is to foster sub-regional and national capacity for climate information, prediction products and services, early warning, and related applications for sustainable development in the IGAD region. ICPAC has computer services and data management, climatology, climate diagnostics, prediction, climate applications, documentation, research and development, and end-user liaisons, including a designated lead in each partner country. 49 ICPAC works with the National Meteorological and Hydrological Services (NMHSs) of its Member states as well as the World Meteorological Organization and other research institutions to address regional challenges of climate-related risks including climate change. ICPAC is also working to develop capacity for long-term climate change projections, but its specialty in seasonal predictions is most relevant for drought and disaster prevention. ICPAC is a partner in the regional USAID/EA environment program, Planning for Resiliency in East Africa through Policy, Adaptation, Research and Economic Development Program (PREPARED). Through PREPARED, USAID/EA provides technical assistance to ICPAC in support of institutional strengthening specifically in the area of early warning, predictions and their related applications. The ICPAC office is in Nairobi. Additional program information can be found at http://www.icpac.net/ . III. Evaluation Objectives USAID support to IGAD has focused on three programmatic areas: 1. CEWARN: USAID/EA has supported CEWARN through its HOA Support Program from 2001 to 2004. Through LSGA No. 623-0009.02 (from September 24, 2006 through December 31, 2015), USAID/EA has supported CEWARN and CEWERUs with the development and implementation of the 2012-2019 CEWARN Strategic Framework. 2. IDDRSI: USAID/EA has supported implementation of the IDDRSI strategy through the PCU for the last two years since November 2012 to build IGAD’s institutional capacity and strengthen knowledge management for IDDRSI. 3. ICPAC: USAID/EA has supported ICPAC under the PREPARED Program since March 2013 with a focus on three priority areas: i) institutional strengthening, ii) production of relevant geospatial climate applications and products tailored for use within East Africa, and iii) strengthening the interface between climate change information producers and end-users.  This external performance evaluation will help determine what programs and activities of IGAD, CEWARN, IDDRSI, and ICPAC are working well and why, and which are not achieving expected results, and why.  The evaluation will provide USAID/EA with information to guide modifications and mid￾course corrections, if necessary, to help IGAD and CEWARN, IDDRSI, and ICPAC improve performance and achieve results.  The evaluation should provide pertinent information, statistics, and judgments to assist USAID, IGAD, and other interested parties (such as other donors) in better understanding the opportunities and challenges faced by IGAD, CEWARN, IDDRSI, and ICPAC in achieving results, and in evaluating the effectiveness of USAID’s support to-date. IV. Audience and Intended Users  IGAD Secretariat  IGAD institutions including CEWARN, IDDRSI Platform, and ICPAC  IGAD Institutional Strengthening Action Plan (ISAP) Coordination Group  Member states  USAID/Kenya  USAID/Washington (particularly the Resilience Secretariat)  USAID East Africa Bilateral Missions (especially Kenya, Ethiopia and Somalia) V. Evaluation Questions Overall: 1. To what extent do the various IGAD efforts supported by USAID coordinate to achieve IGAD’s overall objectives? How could these efforts better complement each other to achieve IGAD’s overall objectives? 50 2. To what extent are current donor structures/relationships conducive to IGAD achieving its organizational objectives and how can these be improved? What are the primary areas USAID should focus on to effectively advance IGAD’s agenda? 3. To what extent are the CEWARN, IDDRSI and ICPAC activities sustainable? What is the extent of IGAD member states’ financial, political and in-kind support for these activities and how can that support be strengthened? 4. What has been IGAD experience with USAID partnership? Are there areas, which can be improved and if so, which areas are those? Program Specific: CEWARN: 1. To what extent is CEWARN on track to fully implement its Strategic Framework? What has worked well and what has not? Are all CEWERUs equally effective in advancing the Strategic Framework or are there varying levels of success? What factors contribute to any differing levels of success and what lessons can CEWERUs learn from each other? 2. Are CEWARN and its national level CEWERU interventions meaningful and responsive to cross border conflicts that occur in the region? If so, what factors have made that possible? What have been the main challenges for CEWARN and the CEWARUs in addressing cross border conflicts? To what extent can CEWARN be expected to effectively contribute to advancing peace and stability in the East Africa region it serves? 3. To what extent do civil society organizations in the East Africa region work with and support CEWARN and/or the CEWERUs? What recommendations do the regional stakeholders, including CSOs, have for improving the effectiveness of the CEWARN and CEWARU programs? IDDRSI: 1. How has IGAD leveraged USAID support to strengthen the PCU to implement the IDDRSI strategy? 2. Since its inception, how has USAID support developed the foundations for knowledge management in IGAD? What challenges have been experienced by IGAD in operationalization of the knowledge management system? How have these challenges been handled? 3. USAID/EA has supported the institutional capacity strengthening of IGAD by providing resources for IGAD to hire core finance and administration staff. To what extent have the expected results and/or outcomes been achieved? What steps is IGAD taking to assume responsibility for funding of these staff to avoid dependence on USAID? What is the anticipated timeline for IGAD to fully assume funding responsibility for these staff? ICPAC: 1. To what extent has ICPAC been effective and efficient in achieving its strategic objectives and responding to climate change priorities? 2. To what extent are USAID-supported programs responding to thematic priorities as outlined in ICPAC strategic plans and other regional strategies agreed to by member states? 3. How are current systems and processes of ICPAC helping or hindering it from meeting regional climate change demands and priorities? VI. Evaluation Design and Methodology USAID seeks the most robust evaluation design and methodological approach that is appropriate for the scope of the project, resources, and audience. A non-experimental evaluation design is preferred for the evaluation although the offeror is free to propose an alternative design of choice to be agreed upon with the Evaluation Team prior to commencement of the study. 51 A post-award conference will be held to review the Statement of Work, clarify any questions that may arise, and address any concerns related to the selected offeror’s proposal, including the evaluation team, methodology, and implementation timetable. The post-award conference may be held via teleconference, as appropriate. Evidence gathered will be from both primary and secondary sources. Both qualitative and quantitative data will be collected and analyzed for this evaluation. Methodological triangulation is encouraged in this study. An illustrative set of possible methods include the following: a) Secondary data A desk review of key relevant documents (see list below). For example, review of information from Joint Border Posts will be undertaken to understand the impacts of the project on reduction in trade barriers. Content analysis of all available secondary data relevant to the evaluation will also be undertaken. Key Documents to be reviewed will include but are not limited to:  Partner Instruments (Contract, Cooperative Agreements, etc.)  Partner Annual Work Plans  Partner Annual Reports  Prior year USAID Annual Reports  Relevant IGAD and member state documents, such as: o IGAD Institutional Strengthening Action Program (ISAP) reports o National Country Programming Papers o IDDRSI Strategy o ICPAC Self-Assessment 2010 o ICPAC Strategy and Implementation Plan 2010-2013 o CEWARN Strategy 2007-2011 o CEWARN Strategy Framework 2012-2019 o CEWARN Annual Performance Report 2013 o CEWARN Monitoring Evaluation Reporting and Learning Framework 2014-2019  NGO Reports  Evaluations/Assessments of IGAD including but not limited to: o Final Evaluation Report: Evaluation of USAID/EA Assistance to IGAD/CEWARN (Revised April 2009) o GiZ Assessment of IGAD o Canadian International Development Agency (CIDA) 2007 Assessment of IGAD o Peace and Security Strategy Final December 2010 o Finland Evaluation Report Re: ISAP b) Primary data Interviews will be held with staff from CEWARN in Ethiopia, ICPAC in Nairobi, IDDRSI implementation unit and IGAD Secretariat in Djibouti, USAID/East Africa, three CEWERUs in the region, other donors and development partners. Field visits will be made to the Kenya CEWERU, Uganda CEWERU, and Ethiopia CEWERU to gather the required data. Key informant interviews will be held with but not limited to the following:  IGAD and CEWARN managers  USAID/EA personnel  Relevant bilateral missions  USAID Washington Resilience Secretariat  Other development organizations, stakeholders and donors (e.g., GIZ, European Union (EU), Department for International Development (DfID), CIDA)  Civil society organizations in the East Africa region.  Beneficiary community representatives  Meteorological services of EAC member countries  PREPARED project staff 52 The Evaluation Team may provide suggestions on data collection methods and analysis approaches that, in their opinion, best accommodate the objectives of the evaluation. The final evaluation approach will be negotiated with the Mission. Data analysis methods The offeror should propose a clear plan for analyzing and triangulating data from various sources to generate high quality and credible evidence to answer the evaluation questions. The analysis method proposed must be relevant to the data collection tools proposed. Potential limitations of methodologies proposed should be highlighted. All conclusions made by the evaluation team must be supported by clear, verified evidence. Anecdotal evidence will not be considered sufficient for drawing conclusions. VII. Evaluation Team Composition The offeror shall propose the most effective team (or teams) composition based on the proposed methodology, within the budgetary limitations. All team members must have relevant prior experiences in Africa, familiarity with USAID’s objectives, approaches, and operations and prior evaluation/assessment experience. In addition, individual team members should have the technical qualifications identified for their respective positions. The team should have sufficient relevant experience in agribusiness, climate change, conflict early warning and more broadly peace and security, to effectively conduct the evaluation. The following is an illustrative team composition: Evaluation Team Leader and Lead Investigator (M&E/Knowledge Management specialist): The TL is ultimately responsible for the overall management of the evaluation team and the final products. In addition, the TL is responsible for coordinating evaluation activities and ensuring the production and completion of an evaluation report in conformance with this scope of work and timelines. The TL will ensure high quality analysis, writing quality and report integration. S/he is also responsible for quality assurance and timeliness of all deliverables. S/he is responsible for the writing of the final evaluation report and preparing and submitting all Task Order deliverables. All team members report to the Team Leader. Senior Investigator (Organization development specialist): The senior investigator, together with the Team Leader, will finalize the evaluation methodology; develop the data collection strategy, instruments, and protocols; direct data collection and compilation; and conduct data analysis. Local investigator/s: Local investigator/s are responsible for data collection, data compilation, and part of initial data analysis. Local investigator/s may be hired directly or sub-contracted through a local survey firm, public opinion research firm, or other local research institution/organization, as appropriate for the proposed methodology. The offeror is strongly encouraged to consider partnering with a local survey or research firm for data collection, preliminary analysis, and logistical and other support. VIII. Evaluation Deliverables Inception Report: Within five work days of the contract signing, the offeror must submit a detailed inception report to USAID. The report shall detail the evaluation design and operational work plan, which must include the proposed data collection and analysis methods to address the Key Questions of the evaluation. The inception report shall also include questionnaires and interview protocols. Preliminary Draft Evaluation Report: Within five weeks of USAID’s acceptance of the Inception Report, the offeror must submit a draft evaluation report and a power point version to USAID for preliminary comments prior to final Mission debriefing. This will facilitate preparation of a more final draft report that will be left with the Mission upon the Evaluation Team’s departure. 53 Debriefing: Within five weeks of USAID’s acceptance of the Inception Report, and immediately at the close of fieldwork and before the offeror’s team departs East Africa, the team must present the major findings of the evaluation to USAID/EA, IGAD, CEWARN, ICPAC and other partners through a PowerPoint presentation. The debriefing shall include a discussion of findings, conclusions, and recommendations. Interim Evaluation Report: Within 10 work days after the debriefing, the offeror must submit a draft report of the findings, conclusions and recommendations to USAID, including revisions based on USAID/EA and partner comments from the debriefing. The written report must address the evaluation questions; clearly describe findings, conclusions, and recommendations. USAID will provide comment on the draft report within two weeks of submission. Final Report: Within 10 work days of USAID’s comments on the Interim Evaluation Report, and based on the provisions of the USAID evaluation policy, a formal and final evaluation report shall be presented to USAID/EA. The final report shall incorporate the team responses to Mission’s comments and suggestions. The format shall include an executive summary (highlighting key lessons learned), table of contents, list of acronyms, evaluation design and methodology, findings, conclusions, and recommendations and lessons learned. The report shall be submitted in English, in both electronic and three bound hard copies. The Final Report must not be more than 40 pages excluding annexes. The report will be disseminated within USAID. A brief summary of this report (the popular version), not exceeding 15 pages, excluding any potentially procurement￾sensitive information shall be submitted (also electronically, in English) for dissemination among implementing partners and stakeholders. The report must meet standards out-lined in the evaluation policy215 (see check list on page 9 of 26). USAID/EA Program Office shall submit the report to the DEC after approval. IX. Evaluation Management a) Logistical Support: USAID/EA will provide other relevant documents and information for this assignment. USAID/ EA PDI will manage this contract and coordinate with technical officers. The offeror will be fully responsible for all logistical and secretarial support including local and regional travel and will be required to demonstrate ability to obtain any security and medical clearances required by USAID. b) Scheduling: USAID/EA expects this evaluation to take place beginning o/a November 2014 until December 2014. c) Summary of Budget: A budget template is attached which must be used for presentation of the proposed budget to USAID. d) Reporting Guidelines:  The evaluation report should represent a thoughtful, well-researched and well-organized effort to objectively evaluate what worked in the project, what did not and why.  Evaluation reports shall address all evaluation questions included in the scope of work.  The evaluation report should include the scope of work as an Annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by USAID  Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.  Evaluation findings will assess outcomes and impact on males and females. 215 http://www.usaid.gov/evaluation/USAID_EVALUATION_POLICY.pdf?020911 54  Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an Annex.  Recommendations need to be supported by a specific set of findings.  Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. USAID Evaluation Policy standards, found in the link below, must be met by the offeror throughout the contract. http://www.usaid.gov/sites/default/files/documents/1868/USAIDEvaluationPolicy.pdf 55 Annex 3: Evaluation Team Member CVs COLIN REYNOLDS Nationality: Australia Affiliation: IBTCI Position Title: Team Leader Labor Category: Social Scientist Education: M.Sc. Cross-border Economics, University of Queensland 1994 B.Sc. Zooloogy, University of Queensland 1985 Relevant Experience: Mr. Colin Reynolds has extensive experience in peace-building, conflict resolution, disaster risk management and stabilization. His experiences range from Environmental Science and Resource Management to Capacity Building and Monitoring, Evaluation and Design. In the past, Mr. Reynolds has attended various conferences in International Peace Monitoring Team Security, Communications and Conflict Zone Training, RedR Essentials of Humanitarian Practice Course and advanced UN Conflict environment courses. Specifically, Mr. Reynolds has held several positions in Natural Resource Management including Queensland Environmental Protection Agency, Trade Records Analysis of Flora and Fauna in Commerce, United National Development Program and AusAID. He has administered the production of regional management plans, integrate strategies to improve land planning and resource management, organize consultation workshops to review management objectives and implement international legislation and regulatory frameworks. During these activities, Mr. Reynolds has gained substantial experience in countries such as Djibouti, South Africa, Mozambique, Senegal, Botswana, Ethiopia, Kenya, and Malawi. Professional Experience: April 2014 – January 2015, Team Leader, Evaluation Specialist, USAID – Manage Summative Evaluation of Yemen OFDA & FFP humanitarian programs; and design participant surveys and interviews. July 2013 – December 2013, Team Leader, AusAID (Afghanistan) (Coffey International) –Conducted Mid Term Review (field-based) for the ‘Children of Uruzgan’ Basic Health and Education Program. March 2011- September 2011, Stabilization Analytics Specialist, USAID Somalia – Reviewed and revised current stabilization theory in light of world-wide experience, and adapt approaches to the context of Somalia; designed and implemented means of measuring the stabilization effects of the combined USAID program in Somalia (including development of Mission Performance Management Plan [PMP] and associated field research during its implementation); trained USAID implementing partners in design and application of targeted stabilization approaches; and designed and implemented (including the provision of enumerator training) social, political and economic surveys associated with above tasks. August 2010- Jan 2011, Stabilization Analytics Specialist, USAID, Yemen – Reviewed and revised current stabilization theory in light of world-wide experience, and adapt approaches to the context of Yemen; Designed and implemented means of measuring the stabilization effects of the combined USAID program in Yemen (including development of Mission Performance Management Plan [PMP] and associated field research during its implementation); trained USAID implementing partners in design and application of targeted stabilization approaches, including development of training approaches and materials; prepared Quarterly Stabilization Report (QSR); conducted Annual USAID Data Quality Assessments. March 2010-July 2010, Policy Development Consultant, Oxfam Australia – Developed and summarize policy briefs to government in relation to overseas mining activities, Australia’s Export Credit Agency, an international Financial Transactions Tax (FTT), application of the Paris Declaration and Accra Statement, Water and Sanitation (WASH) elements of Australian aid program, Australian engagement in Afghanistan, and rural development. 56 January 2010, Team Leader Evaluation, AusAID, Indonesia – Led the evaluation for Independent Completion Report of the post-earthquake, Disaster Risk Reduction program: Australia Indonesia Partnership for Reconstruction and Development (AIPRD), Yogya-Central Java Community Assistance Program (YCAP). October 2009 –December 2009, Team Leader, UNESCO, Afghanistan – Led the field evaluation of the Enhancement of Literacy in Afghanistan Program (ELA); and developed revised M&E framework for identifying effects of improved literacy. September 2009 – October 2009, Team Leader, Multi Agency (UNTAET-INGO), USAID/AUSIAD Funded, Timor-Leste – Led joint field evaluation of multi-agency (UN & INGO) humanitarian response to the 2006-08 (IDP riots) crisis in Timor-Leste. December 2007 – April 2008, Team Leader (Evaluation/Redesign), USAID, Iraq (IBTCI) – Led a team of specialists to evaluate and redesign USAID Local Governance Program (LGP) in Iraq; and prepare draft PMP for measuring results on ongoing stability and development. August 2007 – October 2007; Evaluation Team Leader, United Nations Development Program (UNDP), Afghanistan – Led participatory evaluation of the Regional Initiative for Sustainable Economies (RISE) [Joint FAO/UN-UDG initiative]; designed improved metrics for analyzing success of RISE economic activities February 2007 – April 2007, Program Design Team Leader, Department for International Development, UK (DFID), Afghanistan – Designed on a consultative basis, Technical Assistance Facility: ‘Strengthening Afghanistan’s Budget Formulation And Execution By National And Provincial Government;’ Conduct participatory Project Completion Review (PCR) on existing DFID ‘Transforming and Modernizing the Budget Formulation Process’ project; and design draft M&E framework for measuring impact of budgetary reforms. December 2006 – January 2007, Evaluation Team Leader, United Nations Development Program (UNDP), Afghanistan – Conducted and manage participatory evaluation of the Making Budgets Work (MBW) Project (UNDP’s ‘flagship’ project for Afghanistan); and redesigned M&E framework for measuring impact of national budgeting reforms January – February 2005, Agriculture Sector Specialist, Asian Development Bank, Afghanistan – Team member on Poverty Assessment and Socioeconomic and Macroeconomic Statistical Capacity Building Project (within Afghan Central Statistics Office). November-December 2004, REA Advisory Group Member, CARE International/UNHCR, Nepal – Undertook field testing (Bhutanese refugee camps in Nepal) of Guidelines for Rapid Environmental Impact Assessment in Disasters. August-September 2004, Evaluator/Monitoring Systems Designer, USAID, East Timor – Undertook an in-country, impact assessment/evaluation of the USAID Program Support Initiative (small grants program) for East Timor (2001-2004); developed PMP indicators and survey techniques applicable to the highly variable context of the program, with the intent of measuring impacts of rural development projects on development and stability issues; and reviewed all projects included in the program. February 2003 - July 2003, Peace Building Consultant, Swisspeace Foundation, Afghanistan –Analyzed drivers of conflict (in a basic systems analysis framework) and develop means of promoting and measuring improved stability within Afghanistan; designed and initiate National, community-based peace building/stability program for Afghanistan, in collaboration with local partners and the Afghan Transitional Authority; conducted rapid field evaluation of Swisspeace operations in Afghanistan; and train local and international staff on conflict/post-conflict security protocols and practices February 2003 – July 2003, Monitoring and Evaluation Consultant /Program Designer (Kabul), US State Department (INL)/USAID, Relief International, Afghanistan – 57 Conducted field evaluation of the Creation and Restoration of Alternate Rural Livelihoods and Security (CRALS I) project in Afghanistan (a US State Department poppy reduction initiative); re￾designed of CRALS II program to incorporate coordination with National Solidarity Program (World Bank) operation and monitoring principles; designed and conduct Evaluation/Monitoring ‘train-the￾trainer’ program for key national and international Relief International staff; conducted a three month Implementation review and development of revised monitoring/evaluation framework design for CRALS II focusing on poppy reduction and stability targets; and presented findings and recommendations to the funding agency (INL) in Washington. August 2002 - September 2002, Evaluation Methodology Specialist/Evaluation Team Member, United Nations Office for the Coordination of Humanitarian Affairs, Afghanistan – Designed and participate in evaluation of OCHA facilitation/coordination of humanitarian assistance to Afghanistan July 2001 to July 2002; devised quantitative/qualitative metrics for assessing intervention impacts on development and stability; and provided Civil-Military Coordination and Peace/Stability Building sector focus. December 2001 - March 2002, Field Team Member, Solomon Islands International Peace Monitoring Team, Solomon Islands – Patrolled remote villages and assess violations of the Townsville Peace Agreement (TPA); accepted weapons surrenders in accordance with the TPA; organized reconciliation events, including the Marau Sound Peace Fair and the Honiara Peace Festival; monitored and evaluated impacts of IPMT initiatives on stability and future conflict resilience; assisted local communities in reconstruction work and environmental resource management initiatives; and engaged local NGOs and assist in development of peace building initiatives October 1996 - July 2000, Principal Planning Officer, Queensland Environmental Protection Agency (EPA) - Designed and implemented data collection methodologies and environmental assessments for coastal planning programs; administered production of Regional Coastal Management Plans; supervised Senior Planning Officers and other Departmental staff; designed and undertook consultation with all levels of government, industry and the general public; designed and implemented integration strategies regarding improvements to land planning and resource management; and undertook field assessments of environmental damage and undertake enforcement work as necessary. March 1995 - March 1996, Associate Adviser, AusAID/Western Samoa Fisheries Extension and Training Project, Management Agent International Development Support Services OXFAM/CAA – Assisted in the preparation of Project Implementation Documentation (for funding approval by AusAID); prepared and undertake village-oriented questionnaires, surveys and associated analyses; recruited, supervised and supported local staff working in region; designed and conducted formal (University accredited) courses and on-the-job training of local staff; organized educative demonstrations and public field-days; liaised with local and international NG0s; and assisted Village fishers and local entrepreneurs in the definition and establishment of fishing and 'downstream' projects January 1994 - October 1996, Regional Planning Officer, Queensland Environmental Protection Agency (EPA) – Assisted in design and implement data collection methodologies for coastal planning programs; provided technical and policy advice regarding the incorporation of environmental considerations into the drafting of the Coastal Protection Bill, 1994, the Land Bill, 1994 and the Fisheries Bill 1994; organized Aboriginal and Torres Strait Islander consultation workshops Review of regional planning options in regard to coastal management objectives; functioned as liaison with other government and non-government agencies with regional planning interests; reviewed planning considerations for State Planning Policies and State and regional coastal management plans; represented EPA on the Queensland Arbovirus* Advisory Committee and Arbovirus Technical Working Group (convened by Queensland Department of Health) (*mosquito-borne diseases); lectured for the Queensland University of Technology in regard to their 58 "Advanced Mosquito Control Course;" and prepared TORs, consultants' briefs and associated on-going liaison. December 1992 - March 1993, Consultant, World Wide Fund for Nature (WWF), Australia –Reviewed WWF policy on sustainable use of wildlife; and assisted in the preparation of a species-based action plans for the Queensland Wet Tropics World Heritage Area. July 1992 - December 1992, Research Analyst, IUCN, TRAFFIC Oceania (Trade Records Analysis of Flora and Fauna in Commerce) – Reviewed Australasian environmental legislation, management plans, and associated documentation; reviewed IUCN Trade policy and related conventions (e.g. CITES); carried out research and prepare detailed reports regarding various issues associated with both legal and illegal trade in wildlife and wildlife products; and reviewed and reported on the implementation of international legislation, regulatory frameworks and conventions (i.e. CITES) for existing trade in Australian and Asia-Pacific wildlife. June 1987 - February 1992, Project Officer/Researcher, National Parks and Wildlife Office of the Sarawak Forest Department in association with the Australian Overseas Service Bureau, Malaysia - Liaised with timber industry in regard to forest conservation initiatives and issues such as Terrestrial and marine wildlife surveys with special emphasis on the training of local staff in survey techniques, Environmental impact assessments, Baseline environmental surveys, Rainforest/marine conservation planning; determined appropriate management strategies to sustain a small traditional industries within reserves; reviewed wildlife and national park legislation from other tropical countries; represented Agency as a full member of Sarawak State Legislative Assembly Special Select Committee on Flora and Fauna; prepared drafting instructions, State Government Bills, Cabinet Papers and Second Readings relating to the revision and amendment of national park and wildlife legislation in Sarawak; prepared draft conservation submission for the Sixth Malaysia Plan; coordinated multi-lateral negotiations between United States, Australian, and Malaysian Government conservation agencies; reviewed the potential hazards posed to the public within the national park and reserve system; led evaluative study mission of international marine research and ecotourism facilities; and reviewed South China Sea oil-spill contingency plans. December 1984 - December 1985, Research Assistant, The University of Queensland, Zoology Department (Fisheries Module) – Assisted in the design, field sampling and data analysis of fisheries field studies. June – July 2004, Evaluator, United Nations Development Program, Uzbekistan – Conducted a field evaluation of the UNDP Atrof-Muhit (Environmental) Program in Uzbekistan. April 2012– January 2014, Monitoring and Evaluation Adviser Australia-New Zealand￾ASEAN Free Trade Agreement Program, AusAID (Asia Regional) - Designed and implemented Monitoring and Evaluation Framework to support implementation of the program; and provision of on-going advice in response to developments encountered during implementation. February 2012 – April 2012, Monitoring and Evaluation Specialist ASEAN Program, AusAID (Asia Regional) – Conducted the ASEAN-Australia Development Cooperation Program Phase Two Mid Term Review; and reviewed economic and trade issues addressed by the program and assess the success of implementation to date. February 2014 – June 2014, Team Member, Department of Foreign Affairs and Trade, Australia – Redesigned South and West Asia Australia Awards Program with the focus on M&E and Security Issues. June 2012 – January 2015, Team Leader, Coffey International, AusAID, Pakistan – Conducted Independent Progress Report (IPR) for Australia Awards in Pakistan Program; and designed, conducted, analyzed and reported on participant surveys and interviews June 2012 – January 2015, Monitoring and Evaluation Adviser South Asia Scholarships Program (SASP), Coffey International, AusAID, South Asia - Designed and implemented 59 Monitoring and Evaluation Framework for the program; and designed, conducted, analyzed and reported on participant surveys and interviews. May 2012 – June 2012, Design, Monitoring and Evaluation Specialist Program, AusAID, Pacific Regional – Field and desk work associated with developing a model for coordinated outsourcing of Australia Awards programs in the Pacific. January 2012 – February 2012, Team Leader Technical Advisory Group (Australia Awards in Africa), AusAID, Africa – General TAG duties including conducting Independent Progress Report (IPR) of Australia Awards in Africa program. January 2012 – February 2012, Design, Monitoring and Evaluation Specialist, GRM, AusAID, Africa – Quality Assurance Systems Review of Australia Awards in Africa Program. December 2011– December 2011, Design, Monitoring and Evaluation Specialist, AusAID, Latin America – Rapid Evaluation of Latin American Australia Awards Program. October 2011 – December 2011, Design, Monitoring and Evaluation Specialist, AusAID – Revised AusAID Scholarships design and M&E guidance to match new ‘Australia Awards’ environment. February 2010 – March 2010, Team Leader Design, AusAID (Various) – Finalized design for revised components of AusAID Scholarships programs in Pakistan and Africa (various countries), including various initiatives to promote and measure improved national leadership and workforce capacity; and fulfilled Gender Specialist role in Africa design mission. August 2009, Team leader, PNG, AusAID – Led the Independent Completion Report of the Papua New Guinea-Australia Targeted Training Facility (PATTAF) May 2008 – August 2009, Design, Monitoring and Evaluation Specialist, AusAID (Global) – Led global review of offshore aspects of AusAID study assistance programs, specifically targeting utilizing and measuring the effects of such programs on longer-term development/stability of recipient nations; conducted associated M&E training for AusAID international staff; and became the Gender and design specialist for redesign of AusAID study assistance programs for Africa, SE Asia and Pakistan, including various initiatives to promote and measure improved national leadership and workforce capacity. May 2007 – June 2007 Consultant Evaluator, AusAID, China & Mongolia – Cluster evaluation of four ANCP-funded NGO projects (field evaluation: China (incl. Tibet)/Mongolia) March 2005 – December 2006, M&E Design Consultant, Australian Agency for International Development (AusAID): Various Country Programs - Independent mediation facilitator (AusAID – Contractor) regarding payment against milestones (North Asia). Conducted meta-evaluation of AusAID-NGO Cooperation Program evaluations 2004-07 (CPS); conducted inaugural Annual Review of AusAID M&E/Quality Assurance Systems; advised on M&E-focused change management issues in regard to AidWorks & White Paper reforms (including scholarship expansion); designed or revise AusAID monitoring and evaluative systems for Performance Information Framework (PIF) Review, with particular reference to measuring long term development or stability outcomes in recipient countries; reviewed activity design documents; functioned as methodology specialist on evaluation of AusAID Tsunami Response; and conducted Mid-Term Review of the Vietnam Australia Monitoring and Evaluation Strengthening Project (VAMESP) as team member with special focus on corporate harmonization/alignment (AusAID is signatory to Paris declaration and Hanoi Core Statement) and IT system analysis. July 2003 – January 2004, Contract Aid Expert (Executive Level 1), Australian Agency for International Development (AusAID), Various Countries – Reviewed the Australian Aid to the Pacific 1975 –2003 specifically in relation to effects of Aid on promoting development and stability (within fragile states); undertook Solomon Islands Conflict Case Study to determine both drivers of that conflict and means of addressing and monitoring them; designed the implementation 60 of data analysis methodologies, including the development of automated monitoring systems software; and reviewed and revised AusAID activity monitoring systems, including improving current approaches to log frame systems and development of new Environmental, Multilateral, Humanitarian, Development Bank, NGO and sectoral monitoring systems July 2000 - February 2003; Monitoring and Evaluation System Designer/Trainer, Australian Agency for International Development (AusAID), Various - Revision and Design of AusAID Activity Monitoring Systems, including: design of new monitoring/evaluation methodologies (SMT), design and delivery of log frame/project cycle training courses at both Canberra and Posts and liaison with other donor agencies as to current monitoring/evaluation approaches. Languages: English (5), and Bahasa (3). References: Dr Carroll Patterson Email: carrollpatterson@gmail.com Phone: +1 302 476 2420 Address: 1201 N. Orange Street Wilmington, Delaware 19801 USA Ms. Rachel Ingwersen Program Manager Australian Department or Foreign Affairs and Trade Email: rachel.ingwersen@dfat.gov.au Phone: +61 2 6178 4671 Jennifer Robinson M&E Systems Architect and Data Quality Specialist Email: jenni@wazima.co.za Fax: +27 86 621 1563 Cell: +27 83 460 7752 61 Name Wepundi Manasseh Juma Nationality: Kenyan Affiliations: IBTCI Position Title: Senior Investigator Labor Category: Monitoring and Evaluation Education: MA International Studies, University of Nairobi, Kenya 2010 BA Government and Public Administration, Moi University, Eldoret, Kenya 2002 Relevant Experience: Manasseh Wepundi has over 13 years’ experience in human security research and programming in Eastern Africa. He has worked with teams of diverse cultures and disciplines in monitoring and evaluating different peace and development programs in the region. He has led different research teams in various projects, a number of which have been published. He has a good command of qualitative and quantitative methodologies. Selected Professional Experience: United Nations Development Program, Nairobi Lead Consultant Oct 2014 – Jul 2015: Developed research instruments, and constituted and led a team of 35 research assistants and six data entrants in baseline research on human rights violations and sexual and gender based violence against women living with HIV (in Kakamega, Kisumu, Mombasa and Nairobi). The study employed qualitative and quantitative methods. Impact Africa Ltd., Nairobi Lead Researcher Nov 2014 – June 2015: Led a team of 16 research assistants and three data analysts in carrying out a baseline survey on environmental awareness in the six coastal counties of Kilifi, Kwale, Lamu, Mombasa, Tana River and Taita Taveta. The themes of study were coastal biodiversity management, shoreline change, waste management and gaps in environmental laws and policies. The baseline study then informed development of an Integrated Coastal Zone Management Communication Strategy, with an implementation plan and monitoring and evaluation framework. Impact Africa Ltd., was contracted by governmental agencies led by the National Environmental Management Authority (NEMA). CODESRIA, Dakar, Senegal Researcher Nov 2014 to present: Part of an inter-disciplinary inter-state Research Network contracted by the Council for the Development of Social Science Research in Africa to research extremism and state capacity in Kenya and Nigeria. The objective was to determine the extent to which extremism and radicalization have been incubated by lack of state capacity to respond to the terrorist threat. PACT, Nairobi Office Research Consultant Nov 2014 – June 2015: Consulted for PACT in reviewing rapid conflict assessments, advised on the research design for a baseline survey of their Somali and Karamoja cluster areas of interventions, and made preliminary analysis of baseline findings and programmatic implications. These tasks were executed for PACT’s PEACE III program, that focuses on peacebuilding, conflict early warning and early response, and strengthening peace structures in Karamoja and Somali cluster areas of Ethiopia, Kenya, Somalia and Uganda. CEWARN, Addis Ababa Ethiopia Evaluation Consultant August 2014: Conducted a rapid, desktop evaluation of Rapid Response Fund (RRF) projects of the Conflict Early Warning and Early Response Mechanism (CEWARN) of the Inter-Governmental Authority on Development (IGAD). SSANSA, Juba, South Sudan Strategy Consultant August 2014: Facilitated South Sudanese consultations on security sector reform and developed a civil society position paper on South Sudan Security Sector Reforms, ahead of IGAD Peace Talks in Addis Ababa, Ethiopia. These consultations were organized by the South Sudan Action Network on Small Arms (SSANSA) 62 Justice Africa, South Sudan Security Sector Governance Consultant July 2014: Advised and assisted in the development of civil society proposals on a new South Sudan Constitution, with a focus on Security Sector Governance. The consultations were held in Gulu, Northern Uganda and were organized by Justice Africa. CEWARN, Addis Ababa, Ethiopia Monitoring and Evaluation Trainer June 2014: Trained staff members of Conflict Early Warning and Early Response Units (CEWERUs) of CEWARN on monitoring and evaluation and participatory training methods. The CEWERU staff members were from Djibouti, Ethiopia, Kenya, South Sudan and Uganda. Management Systems International, Nairobi Office Conflict Mitigation Specialist Nov 2013 – Jan 2014: Team member – bringing on board conflict mitigation and conflict sensitivity knowledge – for the evaluation of six USAID-funded peace programs (PACT, CHF International, Mercy Corps, International Rescue Committee, Catholic Relief Services, and Internews) that aimed at mitigating effects of 2008 electoral violence & preventing a repeat. The programs were worth $42.7million. The team evaluated various conflict mitigation programs including peace messaging, conflict early warning and early response, peace dividends, and community dialogue projects in different parts of Kenya. United Nations Development Program, Nairobi Monitoring and Evaluation Specialist 2013 (February to June): Developed Monitoring and Evaluation system for the Electoral Violence Reduction Initiative; developed M&E indicators; advised stakeholders on integration of gender indicators in conflict early warning; trained partners on M&E; and monitored and evaluated peace projects under UWIANO Platform for Peace. Members of the UWIANO Platform included the National Steering Committee on Peacebuilding and Conflict Management (NSC), National Cohesion and Integration Commission (NCIC), Independent Electoral and Boundaries Commission (IEBC), Peace and Development Network (PEACENET) and the Kenya Partnership for Peace and Security (PfP). CEWARN, Addis Ababa, Ethiopia Handbook Review Consultant Oct 2012 – Dec 2013: Reviewed CEWARN’s Rapid Response Fund Framework Handbook, simplifying the sections for ease of use by CEWERU members of differing skill sets. The field interviews covered Djibouti, Ethiopia, Kenya, and Uganda. Saferworld, Nairobi Baseline Survey Lead Consultant Mar – Jun 2012: Conducted baseline assessments for the Nairobi office of Saferworld. The baseline study was for community policing projects in Bungoma, Isiolo and West Pokot Counties of Kenya. Local Capacities for Peace International Program Manager 2009-2012: Oversaw the institutionalization of the organization, set up organizational programs. I enhanced the organization’s research capacity, with the biggest achievement being a two-year initiative to build the capacity of eight international agencies in conflict sensitive programming. Africa Policy Institute, Nairobi Senior Analyst 2007 – 2008: Provided informed leadership and supervision of all projects in the Uganda Liaison Office. Also did research and analysis of peace and security dynamics in the Horn and prepared situation reports and policy briefs on the Uganda peace process. Africa Leadership Institute, Kampala, Uganda Senior Researcher 2006 – 2007: Offered research expertise to Ugandan regional political leaders engaged in the Juba Peace Talks while documenting progress of the northern Uganda peace process. Prepared policy briefs and situation reports, and engaged in public education drives on progress of peace talks 63 Peace Tree Network, Kenya Chapter Associate Coordinator 2005 – 2006: Coordinated a network of peace building civil organisations in Kenya. The network mainly focused on community conflict management, conflict early warning and early response and peace advocacy. A key achievement was the coordination of the Kenyan chapter’s research on elections and violence in a regional project titled “Bullets to Ballots.” As the Kenyan Chapter Coordinator I sat on the regional steering committee of the network with membership in Burundi, Democratic Republic of Congo, Ethiopia, Kenya, Rwanda, Tanzania, and Uganda. Kenya Action Network on Small Arms and Light Weapons (KANSA) Secretary 2003 – 2004: As employee of SRIC which was the coordinating point of KANSA, I served as secretary to the national civil society network of actors and initiatives against illicit small arms possession and proliferation. The steering committee of the network was (and is) chaired by Africa Peace Forum (APFO). The network brought together international, national, and community based civil society actors. KANSA also embraced use of early warning and early response, community dialogue and peace building, security sector reform and governance, and legal reform to combat the problem of small arms and attendant security challenges. Security Research and Information Centre (SRIC), Nairobi Researcher 2002 – 2006: Worked on research projects on conflict and firearms related crimes in the Horn of Africa and Great lakes Region. Gained wide knowledge of issues related to human security, small arms and light weapons, conflict and terrorism in the Great Lakes Region and the Horn of Africa Languages: English, Kiswahili, 64 Annex 4: Matrix of Questions and Sub-Questions REVIEW SUB-QUESTION TYPE OF EVIDENCE DATA COLLECTION SAMPLING OR SELECTION APPROACH DATA ANALYSIS SOURCE METHOD METHOD OVERALL EVALUATION QUESTION 1: To what extent do the various IGAD efforts supported by USAID coordinate to achieve IGAD’s overall objectives? How could these efforts better complement each other to achieve IGAD’s overall objectives? Analytical ES, I-SEC216, ISC, CPS, DP, PSC,TCEW Document review & KII Purposive Content analysis OVERALL EVALUATION QUESTION 2: To what extent are current donor structures/relationships conducive to IGAD achieving its organizational objectives and how can these be improved? What are the primary areas USAID should focus on to effectively advance IGAD’s agenda? Analytical ES, I-SEC217, ISC, CPS, DP, PSC,TCEW Document review & KII Purposive Content analysis OVERALL EVALUATION QUESTION 3: To what extent are the CEWARN, IDDRSI and ICPAC activities sustainable? What is the extent of IGAD member states’ financial, political and in-kind support for these activities and how can that support be strengthened? Analytical ES, I-SEC, ISC, CPS, PSC,TCEW Dir, ICPAC218, Fin/Admin Document review & KII Purposive Content analysis 216 ES-Executive Secretary, I-SEC-IGAD Secretariat, ISC-IGAD Steering Committee, CPS-Committee of Permanent Secretaries, DP-Developmental partners, PSC-Program Steering Committee, TCEW-Technical Working Group-Early Warning 217 ES-Executive Secretary, I-SEC-IGAD Secretariat, ISC-IGAD Steering Committee, CPS-Committee of Permanent Secretaries, DP-Developmental partners, PSC-Program Steering Committee, TCEW-Technical Working Group-Early Warning 218 Dir, ICPAC-Director, IGAD Climate Prediction and Application Center 65 REVIEW SUB-QUESTION TYPE OF EVIDENCE DATA COLLECTION SAMPLING OR SELECTION APPROACH DATA ANALYSIS METHOD SOURCE METHOD CEWARN-SPECIFIC EVALUATION QUESTION 1: To what extent is CEWARN on track to fully implement its Strategic Framework? What has worked well and what has not? Are all CEWERUs equally effective in advancing the Strategic Framework or are there varying levels of success? What factors contribute to any differing levels of success and what lessons can CEWERUs learn from each other? Analytical TCEW, CCC, CSC, NRI, LCM, FM,CSO, BC219 Document review & KII Purposive Content analysis CEWARN-SPECIFIC EVALUATION QUESTION 2: Are and its national level CEWERU interventions meaningful and responsive to cross border conflicts that occur in the region? If so, what factors have made that possible? What have been the main challenges for CEWARN and the CEWARUs in addressing cross border conflicts? To what extent can CEWARN be expected to effectively contribute to advancing peace and stability in the East Africa region it serves? Analytical TCEW, CCC, CSC, NRI, BC, LCM, FM,CSO Document review & KII Purposive Content analysis CEWARN-SPECIFIC EVALUATION QUESTION 3: To what extent do civil society organizations in the East Africa region work with and support CEWARN and/or the CEWERUs? What recommendations do the regional stakeholders, including CSOs, have for improving Analytical TCEW, CCC, CSC, NRI, LCM, FM, CSO Document review & KII Purposive Content analysis 219 TCEW-Technical Working Group-Early Warning, CCC-Country-CEWARN Coordinators, CEWERU Steering Committee, NRI-National Research Institutes, LCM-Local Committee Members, FM-Field Monitors, CSO-Civil Society Organizations, BC-Beneficiary Committee Representatives. 66 REVIEW SUB-QUESTION TYPE OF EVIDENCE DATA COLLECTION SAMPLING OR SELECTION APPROACH DATA ANALYSIS METHOD SOURCE METHOD the effectiveness of the CEWARN and CEWARU programs? IDDRSI/PCU-SPECIFIC EVALUATION QUESTION 1: How has IGAD leveraged USAID support to strengthen the PCU to implement the IDDRSI strategy? Analytical PCU220- Managers, PSC Document review & KII Purposive Content analysis IDDRSI/PCU-SPECIFIC EVALUATION QUESTION 2: Since its inception, how has USAID support developed the foundations for knowledge management in IGAD? What challenges have been experienced by IGAD in operationalization of the knowledge management system? How have these challenges been handled? Analytical PCU￾Managers, PSC, DP Document review & KII Purposive Content analysis IDDRSI/PCU-SPECIFIC EVALUATION QUESTION 3: USAID/EA has supported the institutional capacity strengthening of IGAD by providing resources for IGAD to hire core finance and administration staff. To what extent have the expected results and/or outcomes been achieved? What steps is IGAD taking to assume responsibility for funding of these staff to avoid dependence on USAID? What is the anticipated timeline for IGAD to fully assume funding responsibility for these staff? Analytical ES, CPS, ISC, PSC, Fin/Admin, PCU-Managers Document review & KII Purposive Content analysis ICPAC-SPECIFIC EVALUATION QUESTION 1: To what extent has ICPAC been effective and efficient in achieving its strategic Analytical ISC, I-SC, TCEW, PSC, Document review & KII Purposive Content analysis 220 PCU-Platform Coordination Unit, PSC-Program Steering Committee 67 REVIEW SUB-QUESTION TYPE OF EVIDENCE DATA COLLECTION SAMPLING OR SELECTION APPROACH DATA ANALYSIS METHOD SOURCE METHOD objectives and responding to climate change priorities Dir-ICPAC, Met-Dept. 221 ICPAC-SPECIFIC EVALUATION QUESTION 2: To what extent are USAID￾supported programs responding to thematic priorities as outlined in ICPAC strategic plans and other regional strategies agreed to by member states? Analytical ISC, I-SEC, TCEW, PSC, Dir, ICPAC, Met￾Dept. Document review & KII Purposive Content analysis ICPAC-SPECIFIC EVALUATION QUESTION 3: How are current systems and processes of ICPAC helping or hindering it from meeting regional climate change demands and priorities? Analytical Dir-ICPAC, Met-Dept., Project-staff Document review & KII Purposive Content analysis 221 Met-Dept.- National Meteorological & Hydrologic Services, Meteorology Department, University of Nairobi 68 Annex 5: Complete List of Documents Reviewed 1. Annex B IGAD Performance Evaluation Question Matrix_edited Dec 14_FINAL.doc 2. Annex C IGAD Implementation Work Plan_edited Dec 14.excel 3. USAID KENYA ESPS Proposal for IGAD Performance Review, March 30, 2014 (Revised).doc 4. Agreement Establishing the Inter-Governmental Authority on Development (IGAD), Assembly of Heads of State and Government, Nairobi, 21 March 1996.pdf 5. Memorandum of Understanding, The Intergovernmental Authority on Development and The International Labour Organization, Addis Ababa, Ethiopia- 12th April 201.pdf 6. Action Plan Developed by The Institute of Certified Public Accountants of Cyprus (ICPAC) February 2014.pdf 7. Summary of CEWARN Quarterly Progress Report April-June 2013.pdf 8. Summary of CEWARN Quarterly Progress Report January-March 2013.pdf 9. CEWARN Country Updates: December 2007 – August 2008, For the Djibouti Side of the Afars￾Isa Cluster.pdf 10. CEWARN Country Updates: January – April 2006, For the Ethiopian Side of the Karamoja Cluster.pdf 11. CEWARN Country Updates: May – August 2006, For the Ethiopian Side of the Karamoja Cluster.pdf 12. CEWARN Country Updates: September – December 2006, For the Ethiopian Side of the Karamoja Cluster.pdf 13. CEWARN Country Updates: January – April 2007, For the Ethiopian Side of the Karamoja Cluster.pdf 14. CEWARN Country Updates: May – August 2007, For the Ethiopian Side of the Karamoja Cluster.pdf 15. CEWARN Country Updates: September – December 2007, For the Ethiopian Side of the Karamoja Cluster.pdf 16. CEWARN Country Updates: January – April 2008, For the Ethiopian Side of the Karamoja Cluster.pdf 17. CEWARN Country Updates: May – August 2008, For the Ethiopian Side of the Karamoja Cluster.pdf 18. CEWARN Country Updates: September – December 2008, For the Ethiopian Side of the Karamoja Cluster.pdf 19. CEWARN Country Updates: January – April 2009, For the Ethiopian Side of the Karamoja Cluster.pdf 20. CEWARN Country Updates: May – August 2009, For the Ethiopian Side of the Karamoja Cluster.pdf 21. CEWARN Country Updates: September – December 2009, For the Ethiopian Side of the Karamoja Cluster.pdf 22. CEWARN Country Updates: January – April 2007, For the Kenyan Side of the Karamoja Cluster.pdf 23. CEWARN Country Updates: May – August 2007, For the Kenyan Side of the Karamoja Cluster.pdf 24. CEWARN Country Updates: September – December 2007, For the Kenyan Side of the Karamoja Cluster.pdf 25. CEWARN Country Updates: January – April 2008, For the Kenyan Side of the Karamoja Cluster.pdf 26. CEWARN Country Updates: May – August 2008, For the Kenyan Side of the Karamoja Cluster.pdf 69 27. CEWARN Country Updates: September – December 2008, For the Kenyan Side of the Karamoja Cluster.pdf 28. CEWARN Country Updates: January – April 2009, For the Kenyan Side of the Somalia Cluster.pdf 29. CEWARN Country Updates: September – December 2009, For the Kenyan Side of the Karamoja Cluster.pdf 30. CEWARN Country Updates: January – April 2007, For the Kenyan Side of the Somalia Cluster.pdf 31. CEWARN Country Updates: May – August 2007, For the Kenyan Side of the Somalia Cluster.pdf 32. CEWARN Country Updates: September – December 2007, For the Kenyan Side of the Somalia Cluster.pdf 33. CEWARN Country Updates: January – April 2008, For the Kenyan Side of the Somalia Cluster.pdf 34. CEWARN Country Updates: May – August 2008, For the Kenyan Side of the Somalia Cluster.pdf 35. CEWARN Country Updates: September – December 2008, For the Kenyan Side of the Somalia Cluster.pdf 36. CEWARN Country Updates: January – April 2009, For the Kenyan Side of the Somalia Cluster.pdf 37. CEWARN Country Updates: September – December 2009, For the Kenyan Side of the Somalia Cluster.pdf 38. CEWARN Annual Activity Report December 2009 - November 2010.pdf 39. CEWARN Annual Activity Report 1 November 2007 – 15 December 2008.doc 40. CEWARN Annual Program Performance Report January - December 2013.pdf 41. CEWARN Annual Summary Program Performance Report: Jan – Dec 2012.pdf 42. IGAD Annual Report 2007.pdf 43. IGAD 2009 Annual Report.pdf 44. IGAD Horn of Africa Joint Planning Cell Annual Report September 2011 - June 2013.pdf 45. Investigating the Potential of Peace Committees in Ethiopia. A Needs Assessment in IGAD CEWARN’s Karamoja and Somali Clusters – 2013.pdf 46. Institutional Assessment of the Intergovernmental Authority for Development (IGAD). Assessment Report, July 2006.pdf 47. Evaluation of USAID/East Africa’s (USAID/EA) Assistance To IGAD/Conflict Early Warning and Response Mechanism (CEWARN). Final Evaluation Report, July 2008 – November 2008 (Revised April 2009).pdf 48. Evaluation of IGAD/Conflict Early Warning and Response Mechanism (CEWARN) for USAID/EA/RCMG. Final Report, July 2008 - March 2009.doc 49. Paul Sifverberg, Finland’s Support to Institutional Strengthening of IGAD 2011 – 2014, Mid-Term Evaluation. Draft 15.5.2013.doc 50. Intergovernmental Authority on Development. Organization Structure Review. Final Draft Report, June 2010.pdf 51. Eloise Burke, ICPAC Organizational Self-Assessment, 4 March 2010.doc 52. IGAD Systems audit and IPSAS compliance review.pdf 53. Mid-Term Evaluation for Conflict Prevention and Peace Building Program, Final Report, Addis Ababa, Ethiopia, April 2014.pdf 54. Final Program Evaluation Africa Adaptation program (AAP), 8 April 2013.pdf 55. CEWARN Monitoring Evaluation Reporting and learning (MERL) Framework, 2014 – 2019.pdf 56. Working Paper no. 59 – regional and Global Axes of Conflict- Peacemaking in the Midst of War: An Assessment of IGAD’s Contribution to Regional Security, November 2009.pdf 57. Dolly Afun-Ogidan and Frauke de Weijer. Regional approaches to food security in Africa. The CAAP and other relevant policies and programs in IGAD. Discussion Paper-No. 128e-october 2012.pdf 58. Yufnalis Okubo. IGAD Regional Platform for Ending Drought Emergencies. International Disaster Assistance in the Horn of Africa, 19 April 2012.ppt 59. The CEWARN 2012-2019 Strategy Framework.pdf 70 60. CEWARN Strategy - 2007-2011. CEWARN UNIT, Addis Ababa, Ethiopia, November 2006.pdf 61. Presenting The CEWARN Strategy Framework (Abridged version), 2012-2019 (summary version).pdf 62. Conference to Launch an IGAD Strategy on Peace and Security. Consensus Document and Way Ahead. Khartoum, 1-3 October 2005.pdf 63. IGAD Climate Prediction and Applications Centre (ICPAC) Strategy and Implementation Plan 2010-2013, August 2010.pdf 64. Intergovernmental Authority on Development (IGAD). The IDDRSI Strategy. January 2013.pdf 65. IGAD Food Security Strategy 2005-2008.pdf 66. RRF An Instrument To Support Early Response. CEWARN Protocol Mandate. pdf 67. Intergovernmental Authority on Development (IGAD). Policy Framework For The Establishment of the Eastern Africa Standby Brigade (EASBRIG), Jinja, Uganda 13-16th February 2014.pdf 68. The Conflict Early Warning and Response Mechanism for the IGAD region. pdf 69. Partnerships to implement IDDRSI.pdf 70. Protocol on the Establishment of a Conflict Early Warning and Response Mechanisms for IGAD Member States.pdf 71. Annex V. Conflict Early Warning & Response Mechanism (CEWARN).Report of the Proceedings and Recommendations of the 7th Meeting of the 7th Meeting of the Technical Committee on Early Warning and Response (TCEWR) of CEWARN. 30 November – 1 December 2009. pdf 72. Bernard Bashaasha, Michael Waithaka, Miriam Kyotalimye. Climate change vulnerability, impact and adaptation strategies in agriculture in Eastern and Central Africa: a synthesis. September 2010.pdf 73. CEWARN Quarterly. Empowering stakeholders to prevent violent conflict. May – August 2012, Issue No. 36.pdf 74. Peter Ewell, USAID/East Africa. Drought Resilience and Growth in the Horn of Africa.ppt 75. Intergovernmental Authority on Development. IGAD’s Regional Perspective in Disaster Risk Reduction.doc 76. Intergovernmental Authority on Development (IGAD) & the United Nations Economic Commission for Africa (UNECA). Report of the Launch of the IGAD Sustainable Tourism Master Plan and Meeting of the IGAD Region Tourism Ministers, From 09 – 11 December 2013.pdf 77. East African Community Secretariat. Report of the Stakeholders Preparatory Meeting for the Special Summit of EAC Heads of State on Food Security and Climate Change. Naura Springs Hotel, Arusha, Tanzania, 25th – 26th January, 2010.pdf 78. Stock Taking of Adaptation Activities In The Nile River Basin, October 2010.pdf 79. South Sudan: Keeping Faith with the IGAD Peace Process. Africa Report Number 228. 27 July 2015.Document 80. Stock Taking of Adaptation activities in the Nile RB.pdf 81. Partnerships to implement IDDRSI.pdf 82. IGAD Website: http://igad.int/index.php?option=com_content&view=article&id=93&Itemid=153&limitstart=1 83. IGAD Website: http://igad.int/index.php?option=com_content&view=article&id=93&Itemid=153&limitstart=1 84. ICPAC Website: http://rcc.icpac.net/contents.php?sm_id=1&tm_id=1&cdepth=2&upnum=2&ca_id=3&s1=2&t1=2 85. IDDRSI Strategy, January 2013.pdf 86. IGAD Website: http://resilience.igad.int/index.php/about/strategy 87. CEWARN Website: http://www.cewarn.org/index.php?option=com_content&view=article&id=51&Itemid=53 88. IGAD Website: https://www.google.com/?gws_rd=ssl#q=achievements+for+igad 71 89. Assessment of Economic Integration in IGAD. The Horn Economic and Social policy Institute (HESPI). Policy Papers No. 13/2, August 2013.pdf 90. Rania Hassan (2013) CEWARN’s new strategy framework, African Security Review, 22:2, 26-38, DOI: 10.1080/10246029.2013.792548 91. The Vision of Government of Southern Sudan Southern Sudan Peace Commission (SSPC). Fact Sheet 92. Centre for Basic Research. Publication. Document 93. International Peace Support Training Centre, Nairobi, Kenya. Issue Briefs 2014 Series. Issue No. 7, Fourth Quarter. Document 94. Donatien Nduwimana. The Role of IGAD in Security Stabilization in Somalia and South Sudan. Issue Briefs: Issue No 7. Document 95. The Republic of Uganda. Operational Guidelines for Uganda Conflict Early Warning and Early Response Unit (CEWERU), June 2010. Document The Republic of Uganda. Strategic Plan for Uganda, Conflict Early Warning and Early Response Unit, CEWERU 2012/2017, Aug 72 Annex 6: Responses to Information Requests All the organization listed were asked for formal information request of the most up to date and comprehensive records. No. Organization Request Sent to: Due Date Report Status 1 CEWARN Djibouti Ilmi Awaleh 16-Sep Not received 2 CEWARN Ethiopia Bizusew Mersha 16-Sep Not received Re-sent to Mr. Tesasellassie, Sisay & Ms. Haddis 16-Sep Not received 3 CEWARN South Sudan Taban Charles 16-Sep Not received 4 CEWARN Uganda Joseph Muhumuza 16-Sep Not received Not received 5 CEWERU Djibouti Moussa Mohamed Omar 16-Sep Received 6 CEWERU Ethiopia Tesfaselassie Mezgebe 16-Sep Received 7 CEWERU Kenya Dickson Magotsi 16-Sep Not received 8 CEWERU South Sudan Taban Charles 16-Sep Not received 9 CEWERU Uganda Makmot Okello 16-Sep Not received 10 ICPAC Dr. Guleid Arten 16-Sep Received 11 IDDRSI Dr. John Kabayo 16-Sep Not received 12 IGAD Amb. Mahboub/Sizer 16-Sep Not received ` 73 Annex 7: Instruments Combined KII Instrument: 52 questions selectively drawn from a ‘Question Bank’ based on which respondent was interviewed. Question Bank Question Type: Questions (Ref. No.) Extra Header Data H1. H2. General Application X1a. X1b. X2a. X2b. X3a. X3b. X4a.X4b. X5a.X5b. X6a.X6b. X7a. X7b. X8a. X8b. X9a. X9b.X9c. (X9a1. X9b1.X9c1. X9a2. X9b2. X9c2 X9a3. X9b3. X9c3.) X10a.X10b. X11. (X11i. X11ii, X11iii) X12. CEWARN Specific SC1. SC2. SC3. SC4a. SC4b. SC4c. SC5a. SC5b. SC6a. SC6b. SC6c. SC7. IDDRSI/PCU Specific SPCU1a. SPCU1b. SPCU2a. SPCU2b. SID1a. SID1b. SID3. SID2a. SID2b. SID2c. Y1. Y2. Y3 ICPAC Specific SIC1a. SIC1b. SIC3. SIC2a. SIC2b. SIC2c. Question Application (by Respondent Type) Respondent Type: Questions Applied (Ref. No.) IGAD Secretariat Staff H1. X1a. X1b. X2a. X2b. X3a. X3b. X4a.X4b. X5a.X5b. X6a.X6b. X7a. X7b. X11. IGAD Ambassador Committee Members H1. X1a. X1b. X3a. X3b. X8a. X8b. X9a. X9b.X9c. X11. CEWARN Staff H1. X1a. X1b. X2a. X2b. X3a. X3b. X4a.X4b. X5a.X5b. X6a.X6b. X7a. X7b. X11. CEWERU Staff H1. X1a. X1b. X2a. X2b. X3a. X3b. X4a.X4b. X6a.X6b. X7a. X7b. X11. CEWARN/CEWERU LPUs (Monitors/Committees) H1. SC1. SC2. SC3. National Research Institutes/Financial Managers H1. SC4a. SC4b. SC4c. X11. IDDRSI PCU Staff H1. X1a. X1b. X2a. X2b. X3a. X3b. X4a.X4b. X5a.X5b. X6a.X6b. S1a. S1b. S2a. S2b. X7a. X7b. X11. IDDRSI Beneficiaries (If we can find some) H1. Y1. Y2. Y3. ICPAC Staff H1. X1a. X1b. X2a. X2b. X3a. X3b. X4a.X4b. X5a.X5b. X6a. X6b. X7a. X7b. X11. PARTNER (*THE SPECIFIC USAID PROGRAM*) Staff H1. X1a. X1b. X3a. X3b. X8a. X8b. X11. Other Key CSOs (Do they use services/products?) H2. H1. SC5a. SC5b. SC6a. SC6b. SC6c. SID1a. SID1b. SID2a. SID2b. SID2c. SIC1a. SIC1b. SIC2a. SIC2b. X11. SIC2c.X9a. X9b.X9c. X11. National Government Agencies H2. H1. SC5a. SC5b. SC7. SC6a. SC6b. SC6c. X9a. X9b.X9c. SIC1a. SIC1b. SIC3. SIC2a. SIC2b. SIC2c. SID1a. SID1b. SID3. SID2a. SID2b. SID2c. X12. 74 EAC Staff H1. SC5a. SC5b. SC7. SC6a. SC6b. SC6c. X9a. X9b.X9c. SIC1a. SIC1b. SIC3. SIC2a. SIC2b. SIC2c. SID1a. SID1b. SID3. SID2a. SID2b. SID2c. X11. Other Donor Staff + USAID Staff H1. X10a.X10b. SC5a. SC5b. SC7. SC6a. SC6b. SC6c. X9a. X9b.X9c. SIC1a. SIC1b. SIC3. SIC2a. SIC2b. SIC2c. SID1a. SID1b. SID1c. SID2a. SID2b. SID2c. X11. Draft KII Separated Instruments (Interviews/Group Interviews) Variations by Respondent types: 1. IGAD Secretariat Staff 2. CEWARN Staff 3. CEWERUs Staff 4. IDDRSI PCU Staff 5. ICPAC Staff 6. PARTNER Staff 7. IGAD Ambassadors Committee (DJIB) 8. CEWARN/CEWERU LPU (Committees/Monitors) 9. NRI/FM 10. Other Key CSOs 11. National Government Partners 12. EAC Staff 13. Other Donor Staff + USAID Staff 14. IDDRSI Beneficiaries 75 Instrument 1. IGAD Secretariat Staff Interview Date: Name Position Organisation Sex Location Time For all respondents H1. Please give a brief description of your role at IGAD X1a. From your perspective and experience, what have been the main successes and challenges in implementing each of the following priority sectors of the IGAD Strategy Framework? 1. ‘Agriculture and environment’ 2. ‘Political and humanitarian affairs’ X1b. Do you have any suggestions for improvements? X2a. From your perspective and experience, what have been the main successes and challenges in obtaining/maintaining IGAD budgets and meeting expenditure schedules? X2b. Do you have any suggestions for improvements? X3a. From your perspective and experience, what have been the main successes and challenges in developing/implementing IGAD systems or processes? X3b. Do you have any suggestions for improvements? X4a.From your perspective and experience, what have been the main successes and challenges in developing/maintaining coordination between IGAD programs like CEWARN/CEWERU, IDDRSI and ICPAC? X4b. Do you have any suggestions for improvements? X5a.From your perspective and experience, what have been the main good or bad points about working with foreign donors? X5b. Do you have any suggestions for improvements? X6a.From your perspective and experience, what have been the main good or bad points about member state support to IGAD (Financial/In-kind/Political)? X6b. Do you have any suggestions for improvements? X7a. From your perspective and experience, what have been the main successes and challenges in working with USAID as a funding or implementation partner? X7b. Do you have any suggestions for improvements? 76 Instrument 2. CEWARN Staff Interview Date: Name Position Organisation Sex Location Time For all respondents H1. Please give a brief description of your role at CEWARN. X1a. From your perspective and experience, what have been the main successes and challenges in implementing any of the nine Strategic Pillars of the 2012-2019 CEWARN Strategic Framework? D1. Expanding the coverage of the typologies, causes of violent conflict and geographic areas of focus D2. Organizational and network capability (CEWARN Unit, CEWERUs, Research Institutes and other primary and secondary partners) D3. Network engagement and communications D4. Research and analysis D5. Supporting and scaling response initiatives D6. Learning to identify impact and make relevant adjustments D7. Financial and administrative resources and systems D8. Data quality D9. Decision quality X1b. Do you have any suggestions for improvements? X2a. From your perspective and experience, what have been the main successes and challenges in obtaining/maintaining CEWARN budgets and meeting expenditure schedules? X2b. Do you have any suggestions for improvements? X3a. From your perspective and experience, what have been the main successes and challenges in developing/implementing IGAD or CEWARN systems or processes? X3b. Do you have any suggestions for improvements? X4a. From your perspective and experience, what have been the main successes and challenges in developing/maintaining coordination between IGAD programs like CEWARN/CEWERU, IDDRSI and ICPAC? X4b. Do you have any suggestions for improvements? X5a. From your perspective and experience, what have been the main good or bad points about working with foreign donors? X5b. Do you have any suggestions for improvements? X6a. From your perspective and experience, what have been the main good or bad points about member state support to IGAD or CEWARN (Financial/In-kind/Political)? X6b. Do you have any suggestions for improvements? 77 X7a. From your perspective and experience, what have been the main successes and challenges in working with USAID as a funding or implementation partner? X7b. Do you have any suggestions for improvements? Instrument 3. CEWERU Staff Interview Date: Name Position Organisation sex Location Time For all respondents H1. Please give a brief description of your role at the CEWERU. X1a. From your perspective and experience, what have been the main successes and challenges in implementing your CEWERU Strategy (or equivalent)? X1b. Do you have any suggestions for improvements? X2a. From your perspective and experience, what have been the main successes and challenges in obtaining/maintaining CEWERU budgets and meeting expenditure schedules? X2b. Do you have any suggestions for improvements? X3a. From your perspective and experience, what have been the main successes and challenges in developing/implementing IGAD or CEWARN systems or processes? X3b. Do you have any suggestions for improvements? X4a. From your perspective and experience, what have been the main successes and challenges in developing/maintaining coordination between IGAD programs like CEWARN/CEWERU, IDDRSI and ICPAC at national levels? X4b. Do you have any suggestions for improvements? X5a. From your perspective and experience, what have been the main good or bad points about working with foreign donors? X5b. Do you have any suggestions for improvements? X6a. From your perspective and experience, what have been the main good or bad points about national support to the CEWERU (Financial/In-kind/Political)? X6b. Do you have any suggestions for improvements? X7a. From your perspective and experience, what have been the main successes and challenges in working with USAID as a funding or implementation partner? X7b. Do you have any suggestions for improvements? 78 Instrument 4. IDDRSI/PCU ‘Staff’ Interview Date: Name Position Organisation sex Location Time For all respondents H1. Please give a brief description of your role at IDDRSI. X1a. From your perspective and experience, what have been the main successes and challenges in implementing seven Priority Intervention Areas of the 2013 IDDRSI Strategy? PIA 1: Natural Resources and Environment Management PIA 2: Market Access, Trade and Financial Services PIA 3: Livelihood support and Basic Social services PIA 4: Disaster Risk Management, Preparedness and Effective Response PIA 5: Research, Knowledge Management and Technology Transfer PIA 6: Conflict Prevention, Resolution and Peace Building PIA 7: Coordination, Institutional Strengthening and Partnerships X2a. From your perspective and experience, what have been the main successes and challenges in obtaining/maintaining IDDRSI/PCU budgets and meeting expenditure schedules? X2b. Do you have any suggestions for improvements? X3a. From your perspective and experience, what have been the main successes and challenges in developing/implementing IGAD or IDDRSI/PCU systems or processes? X3b. Do you have any suggestions for improvements? X4a. From your perspective and experience, what have been the main successes and challenges in developing/maintaining coordination between IGAD programs like CEWARN/CEWERU, IDDRSI and ICPAC? X4b. Do you have any suggestions for improvements? X5a. From your perspective and experience, what have been the main good or bad points about working with foreign donors? X5b. Do you have any suggestions for improvements? X6a.From your perspective and experience, what have been the main good or bad points about member state support to IDDRSI/PCU (Financial/In-kind/Political)? X6b. Do you have any suggestions for improvements? SPCU1a. From your perspective and experience, what have been the main successes and challenges of strengthening the PCU? SPCU1b. Do you have any suggestions for improvements? 79 SPCU2a. From your perspective and experience, what have been the main successes and challenges of strengthening IDDRSI and PCU knowledge management? SPCU2b. What steps have been taken to address the challenges? X7a. From your perspective and experience, what have been the main successes and challenges in working with USAID as a funding or implementation partner? X7b. Do you have any suggestions for improvements? Instrument 5. ICPAC Staff Interview Date: Name Position Organisation sex Location Time For all respondents H1. Please give a brief description of your role at ICPAC. X1a. From your perspective and experience, what have been the main successes and challenges in implementing any of the three priority areas agreed with the USAID PREPARED Program. 1. ‘Institutional strengthening’ 2. ‘Production of relevant geospatial climate applications and products tailored for use in East Africa’ 3. ‘Strengthening the interface between climate change information producers and end users’ X1b. Do you have any suggestions for improvements? X2a. From your perspective and experience, what have been the main successes and challenges in obtaining/maintaining ICPAC budgets and meeting expenditure schedules? X2b. Do you have any suggestions for improvements? X3a. From your perspective and experience, what have been the main successes and challenges in developing/implementing IGAD or ICPAC systems or processes? X3b. Do you have any suggestions for improvements? X4a. From your perspective and experience, what have been the main successes and challenges in developing/maintaining coordination between IGAD programs like CEWARN/CEWERU, IDDRSI and ICPAC? X4b. Do you have any suggestions for improvements? X5a. From your perspective and experience, what have been the main good or bad points about working with foreign donors? X5b. Do you have any suggestions for improvements? 80 X6a. From your perspective and experience, what have been the main good or bad points about member state support to IGAD or ICPAC (Financial/In-kind/Political)? X6b. Do you have any suggestions for improvements? X7a. From your perspective and experience, what have been the main successes and challenges in working with USAID as a funding or implementation partner? X7b. Do you have any suggestions for improvements? Instrument 6. PREPARED Staff Interview Date: Name Position Organisation sex Location Time For all respondents H1. Please give a brief description of your role at PREPARED. X1a. From your perspective and experience, what have been the main successes and challenges in implementing any of the three priority areas agreed with the USAID PREPARED Program. 1. ‘Institutional strengthening’ 2. ‘Production of relevant geospatial climate applications and products tailored for use in East Africa’ 3. ‘Strengthening the interface between climate change information producers and end users’ X1b. Do you have any suggestions for improvements? X3a. From your perspective and experience, what have been the main successes and challenges in implementing/dealing with IGAD or ICPAC systems or processes? X3b. Do you have any suggestions for improvements? X8a. From your perspective and experience, what are (other) main strengths and weaknesses of IGAD or ICPAC? X8b. Do you have any suggestions for improvements? Instrument 7. Ambassador/Permanent Sec. /Steering Committee Members Interview Date: Name Position Organisation sex Location Time For all respondents 81 H1. Please give a brief description of your role as a committee member. X1a. From your perspective and experience, what have been the main successes and challenges in implementing the ‘agriculture and environment’ and ‘political and humanitarian affairs’ priority sectors of the IGAD Strategy (especially any related to CEWARN, IDDRSI or ICPAC)? X1b. Do you have any suggestions for improvements? X3a. From your perspective and experience, what have been the main successes and challenges in implementing/dealing with IGAD systems or processes? X3b. Do you have any suggestions for improvements? X8a. From your perspective and experience, what are (other) main strengths and weaknesses of IGAD/CEWARN/IDDRSI/ICPAC? X8b. Do you have any suggestions for improvements? X9a. Which IGAD products or services does your member state or agency use (including any from CEWARN, IDDRSI or ICPAC)? X9b. Which products/services of IGAD do you find most useful? Why? X9c. Are there any important gaps in IGAD products or services (including any under CEWARN, IDDRSI or ICPAC)? Instrument 8. CEWARN/CEWERU LPUs (Monitors/Committees) Interview Date: Name Position Organisation sex Location Time For all respondents H1. Please give a brief description of your role as a monitor or committee member. SC1. What conflicts have occurred (or nearly occurred) in your area (please give brief description and approximate dates)? SC2. What things have been done to help prevent or lessen conflicts in your area? Did they help? Why or why not? SC3. What else could be done to help prevent or lessen conflicts in your area? Instrument 9. National Research Institutes/Financial Managers Interview Date: 82 Name Position Organisation sex Location Time For all respondents H1. Please give a brief description of your role with the NRI or FM. SC4a. From your perspective and experience, what have been the main successes and challenges in working with CEWARN or CEWERUs? SC4b. Do you have any suggestions for improvements? SC4c. Do you have any suggestions for other ways you could assist CEWARN or CEWERUs? Instrument 10. Other Key CSOs Interview Date: Name Position Organisation sex Location Time For all respondents H1. Please give a brief description of the role of your CSO. H2. Please give a brief description of your personal role in the CSO. Set 1. SC5a. Has your CSO been involved in any CEWARN/CERWERU programs (Y/N)? If no, jump to Set 2. SC5b. If Yes, please list the ways you were involved (include locations and dates) SC6a. From your perspective and experience, what have been the main successes and challenges for CSOs who would like to work with CEWARN or CEWERUs? SC6b. Do you have any suggestions for improvements? SC6c. Do you have any suggestions for new ways CSOs like yours could assist CEWARN or CEWERUs? X9a1. Which CEWARN products or services does your CSO use? X9b1.Which products/services of CEWARN do you find most useful? Why? X9c1. Are there any important gaps in CEWARN products or services? Set 2. SID1a. Has your CSO been involved in any IDDRSI programs (Y/N)? If no, jump to Set 3. SID1b. If Yes, please list the ways you were involved (include locations and dates) 83 SID2a. From your perspective and experience, what have been the main successes and challenges for CSOs who would like to work with IDDRSI? SID2b. Do you have any suggestions for improvements? SID2c. Do you have any suggestions for new ways CSOs like yours could assist IDDRSI? X9a2. Which IDDRSI products or services does your CSO use? X9b2. Which products/services of IDDRSI do you find most useful? Why? X9c2. Are there any important gaps in IDDRSI products or services? Set 3. SIC1a. Has your CSO been involved in any ICPAC programs (Y/N)? If no, end. SIC1b. If Yes, please list the ways you were involved (include locations and dates) SIC2a. From your perspective and experience, what have been the main successes and challenges for CSOs who would like to work with ICPAC? SIC2b. Do you have any suggestions for improvements? SIC2c. Do you have any suggestions for new ways CSOs like yours could assist ICPAC? X9a3. Which ICPAC products or services does your CSO use? X9b3. Which products/services of ICPAC do you find most useful? Why? X9c3. Are there any important gaps in ICPAC products or services? Instrument 11. National Government Agencies Interview Date: Name Position Organisation sex Location Time For all respondents H1. Please give a brief description of the role of your Agency. H2. Please give a brief description of your day to day role in the Agency. Set 1. SC5a. Has your Agency been involved in any CEWARN/CERWERU programs (Y/N)? If no, jump to Set 2. SC5b. If Yes, please list the ways you were involved (include locations and dates) SC7. Which products or services of CEWARN or CEWERUs align with needs of your agency’s regional peace and conflict mitigation priorities? 84 SC6a. From your perspective and experience, what have been the main successes and challenges for national government agencies who would like to work with CEWARN or CEWERUs? SC6b. Do you have any suggestions for improvements? SC6c. Do you have any suggestions for new ways agencies like yours could assist CEWARN or CEWERUs? X9a1. Which CEWARN products or services does your agency use? X9b1. Which products/services of CEWARN do you find most useful? Why? X9c1. Are there any important gaps in CEWARN products or services? Set 2. SIC1a. Has your Agency been involved in any ICPAC programs (Y/N)? If no, jump to Set 3. SIC1b. If Yes, please list the ways you were involved (include locations and dates) SIC3. Which products or services of ICPAC align with the needs of your agency’s regional climate change priorities? SIC2a. From your perspective and experience, what have been the main successes and challenges for national government agencies who would like to work with ICPAC? SIC2b. Do you have any suggestions for improvements? SIC2c. Do you have any suggestions for new ways agencies like yours could assist ICPAC? X9a3. Which ICPAC products or services does your agency use? X9b3. Which products/services of ICPAC do you find most useful? Why? X9c3. Are there any important gaps in ICPAC products or services? Set 3. SID1a. Has your Agency been involved in any IDDRSI/PCU programs (Y/N)? If no, end. SID1b. If Yes, please list the ways you were involved (include locations and dates) SID3. Which products or services of IDDRSI align with the needs of your agency’s regional drought and other climate priorities? SID2a. From your perspective and experience, what have been the main successes and challenges for national government agencies who would like to work with IDDRSI? SID2b. Do you have any suggestions for improvements? SID2c. Do you have any suggestions for new ways agencies like yours could assist IDDRSI? X9a2. Which IDDRSI products or services does your agency use? X9b2.Which products/services of IDDRSI do you find most useful? Why? X9c2. Are there any important gaps in IDDRSI products or services? 85 Instrument 12. EAC Staff Interview Date: Name Position Organisation sex Location Time For all respondents H1. Please give a brief description of your role at EAC. Set 1. SC5a. Have you been involved in any CEWARN/CERWERU programs (Y/N)? If no, jump to Set 2. SC5b. If Yes, please list the ways you were involved (include locations and dates) SC7. Which products or services of CEWARN or CEWERUs align with needs of EAC regional peace and conflict mitigation priorities? SC6a. From your perspective and experience, what have been the main successes and challenges of working with CEWARN or CEWERUs? SC6b. Do you have any suggestions for improvements? SC6c. Do you have any suggestions for new ways EAC could assist CEWARN or CEWERUs? X9a1. Which CEWARN products or services does EAC use? X9b1. Which products/services of CEWARN do you find most useful? Why? X9c1. Are there any important gaps in CEWARN products or services? Set 2. SIC1a. Have you been involved in any ICPAC programs (Y/N)? If no, jump to Set 3. SIC1b. If Yes, please list the ways you were involved (include locations and dates) SIC3. Which products or services of ICPAC align with the needs of EAC regional climate change priorities? SIC2a. From your perspective and experience, what have been the main successes and challenges of working with ICPAC? SIC2b. Do you have any suggestions for improvements? SIC2c. Do you have any suggestions for new ways EAC could assist ICPAC? X9a3. Which ICPAC products or services does EAC use? 86 X9b3. Which products/services of ICPAC do you find most useful? Why? X9c3. Are there any important gaps in ICPAC products or services? Set 3. SID1a. Have you been involved in any IDDRSI/PCU programs (Y/N)? If no, end. SID1b. If Yes, please list the ways you were involved (include locations and dates) SID3. Which products or services of IDDRSI align with the needs of EAC regional drought and other climate priorities? SID2a. From your perspective and experience, what have been the main successes and challenges of working with IDDRSI? SID2b. Do you have any suggestions for improvements? SID2c. Do you have any suggestions for new ways EAC could assist IDDRSI? X9a2. Which IDDRSI products or services does EAC use? X9b2.Which products/services of IDDRSI do you find most useful? Why? X9c2. Are there any important gaps in IDDRSI products or services? Instrument 13. Other Donor Staff + USAID Staff Interview Date: Name Position Organisation sex Location Time For all respondents H1. Please give a brief description of your role at [Donor]. X10a. From your perspective and experience, what have been the main good or bad points about donor relationships with IGAD (especially any relating to CEWARN, IDDRSI or ICPAC)? X10b. Do you have any suggestions for improvements? Set 1. SC5a. Have your programs been involved in any CEWARN/CERWERU programs (Y/N)? If no, jump to Set 2. SC5b. If Yes, please list the ways you were involved (include donor programs, locations and dates) SC7. Which products or services of CEWARN or CEWERUs align with needs of your programs’ regional peace and conflict mitigation priorities? Sc6a. From your perspective and experience, what have been the main successes and challenges of working with CEWARN or CEWERUs? 87 SC6b. Do you have any suggestions for improvements? SC6c. Do you have any suggestions for new ways donor programs could assist CEWARN or CEWERUs? X9a1. Which CEWARN products or services do your programs use? X9b1. Which products/services of CEWARN do you find most useful? Why? X9c1. Are there any important gaps in CEWARN products or services? Set 2. SIC1a. Have your programs been involved in any ICPAC programs (Y/N)? If no, jump to Set 3. SIC1b. If Yes, please list the ways you were involved (include donor programs, locations and dates) SIC3. Which products or services of ICPAC align with the needs of your programs’ regional climate change priorities? SIC2a. From your perspective and experience, what have been the main successes and challenges of working with ICPAC? SIC2b. Do you have any suggestions for improvements? SIC2c. Do you have any suggestions for new ways your programs could assist ICPAC? X9a3. Which ICPAC products or services do your programs use? X9b3. Which products/services of ICPAC do you find most useful? Why? X9c3. Are there any important gaps in ICPAC products or services? Set 3. SID1a. Have your programs been involved in any IDDRSI/PCU programs (Y/N)? If no, end. SID1b. If Yes, please list the ways they were involved (include donor programs, locations and dates) SID3. Which products or services of IDDRSI align with the needs of your programs’ regional drought and other climate priorities? SID2a. From your perspective and experience, what have been the main successes and challenges of working with IDDRSI? SID2b. Do you have any suggestions for improvements? SID2c. Do you have any suggestions for new ways your programs could assist IDDRSI? X9a2. Which IDDRSI products or services do your programs use? X9b2. Which products/services of IDDRSI do you find most useful? Why? X9c2. Are there any important gaps in IDDRSI products or services? 88 Instrument 14. IDDRSI Beneficiaries Interview Date: Name Position Organisation Sex Location Time For all respondents H1. Please give a brief description of how drought or other weather problems have affected you. Y1. What weather problems have occurred in your area (please give brief description and approximate dates)? Y2. What things have been done to help lessen the effects of weather problems in your area? Did they help? Why or why not? Y3. What else could be done to help lessen the effects of climate changes? 89 Annex 8: List of Respondents for the KIIs Name Title Agency/Partner Rene c. Guiraud Deputy Head of Mission UNDP Djibouti Hassan Ali Programme Specialist UNDP Djibouti Osman Saad Said Director General Djibouti Met. Dept. Abdourahman Y. Noir Deputy Director Djibouti Met. Dept. Nima Nouh Hassan Met. Officer Djibouti Met. Dept. Gobeh Noud Niguil HRD Officer Djibouti Met. Dept. Liban Ali Soughueh Met. Staff Djibouti Met. Dept. Devia Youssouf Houssein Media Officer Djibouti Met. Dept. Housla Met. Staff Djibouti Met. Dept. Youssouf Houssein Met. Staff Djibouti Met. Dept. Loula Noud Met. Staff Djibouti Met. Dept. Said Ali Darar Information Officer Djibouti Met. Dept. Charmake Barkadleh Thoussa Met. Staff Djibouti Met. Dept. Lony Ruot Chairman SSUNDE Asiki Umari Finance and Administration Officer SSUNDE Awate Edwina Moses Office Assistant SSUNDE Mohamed Ali Hassan Secretary General Ministry of Foreign Affairs and International Cooperation, Djibouti Daher Had Director Finance IGAD/PCU Josephat Onyarti Finance Department IGAD/PCU Debrework Andarge Director Administration IGAD/PCU Haifa Elmubashhar HR Officer IGAD/PCU Bogale Lemma Director Procurement IGAD/PCU Daher Elmi Houssein PM. Natural Resource Management; Agriculture and Environment Division IGAD/IDDRSI Samuel Zziwa PM Agriculture Livestock and Food Security; Agriculture and Environment Division IGAD/IDDRSI Cynthia Odera CSO/NGO Officer: Agriculture and Environment Division IGAD/IDDRSI Anthony Awira M&E Expert IDDRSI IGAD/IDDRSI Eshete Dejen Fisheries Expert; Agriculture and Environment Division IGAD/IDDRSI Dr. Alfred O. Opere Head of Department University of Nairobi, Meteorological Dept. Dr. Gitau Wilson Professor/Lecturer University of Nairobi, Meteorological Dept. Dr. Opijah Franklin Joseph Professor/Lecturer University of Nairobi, Meteorological Dept. Dr. Oludhe Christopher Professor/Lecturer University of Nairobi, Meteorological Dept. Anne Nyabera Executive Director ACT, Kenya 90 Name Title Agency/Partner Richard Orenyo Head of Finance Operations ACT, Kenya Dominic Ruto Pkalya Conflict expert ACT, Kenya Rose Othieno Executive Director Centre for Conflict Resolution John Fisher Conflict Officer Centre for Conflict Resolution Xavier Ejoyi Project Management Specialist -Conflict USAID Uganda Mark Wilson Program Manager USAID Uganda Kiarie Njuguna Desk Officer CEWERU Susan Owiro-Chege Coordinator Kenya Partnership for Peace and Security Milka Chepkirui Gender and Capacity Development Officer CEWERU Leonard Kyalo Program Coordinator Security Research and Information Centre James Ngului Acting Director Kenya National Focal Point on Small Arms and Light Weapons Tamrat Kebede Executive Director Inter Africa Group Haddis Rebbi Inter Africa Group Ms. Fatma Logistics Officer IGAD Belachew Beyene Regional Expert IGAD Ahmed Habbane Program Manager IGAD Gideon Galu Regional Scientist FEWS NET Chris Shitote Assistant Regional Scientist FEWS NET Dr Katharine Downey Coordinator –Technical Consortium for Building Resilience in The Horn of Africa ILRI Sebhat Negga Executive Director Ethiopian International Institute for Peace and Development Moses Monday John Executive Director Organization for Nonviolence and Development Alfred Okech Deputy Head of Programs(also ex CEWARN Country Coordinator) CRS South Sudan Ayten Anemaw Birhanie Executive Director Peace and Development Center Andrea Ghione Senior Economist Italian Development Cooperation Office Mr. Kogna Dneremata Ex- Field Monitor Mr Wako Qanchora Local Peace Committee/Ex-Field Monitor Hon. Chuol Rambang Luoth Chairperson, CEWERU Steering Committee South Sudan Peace and Reconciliation Commission (MS) Dr Richard Barno Director CEWARN CEWARN Dr Kizito Sabala Political and Security Analyst IGAD Mohamed Mousa Mohamed Director Agriculture and Environment IGAD Daniel Yifru Senior Peace and Security Adviser, Peace and Security Division IGAD El-Sadig A. Abdalla Director, Economic Cooperation and Social Development IGAD 91 Name Title Agency/Partner Peter Ambenje Deputy Director, Forecasting and Administration Kenyan Meteorological Department Prof. Laban Ogallo ICPAC coordinator, UNDP Regional DRR program (also ex-ICPAC Director) UNDP Elijah Mukhala Head of WMO, East Africa WMO Makmot Okello Head of CEWERU, Uganda Maj. James Muhumuza CEWERU Member Ministry of Defence, Uganda Paul Isabirye Director, Networks and observations Canon Joyce Nima CEWERU Member CSO -various Juliet Kanyesigye Executive Director Centre for Basic Research, Uganda Mr. James Ndung'u Project Manager, Arms Control and Policing Saferworld Mr. Rashid Kalayu Kaliche Ex- Field Monitor Mr. Lopeyok Dominic Ex- Field Monitor Mr. Joseph Etengan Program Coordinator Losolia Rehabilitation and Development Association Mr. Job Lomong Ex- Field Monitor Mr. Joachim Loduk Ex- Field Monitor Ms. Irene Chelangat Member of District Peace Committee Local District Administration Mr. Hussein Boya Ex- Field Monitor Mr. Galma Dabaso Local Peace Committee Mr. Charles Lopeyu Ex- Field Monitor Mr. Taban Charles John Focal Person CEWERU Mr. Joseph Muhumuza Country Coordinator CEWERU Mr. Tsegaye Bekele Ex- Field Monitor Mr. Admasu Lokali Ex- Field Monitor Helen Hailu Project Officer, Peace and Security Division IGAD Edmond Yakani Executive Director Community Empowerment for Progress Organization (CEPO) Makmot Okello CEWERU Head CEWERU Scott McCormick Chief of Party PREPARED Project Stephen Wathome Program Manager European Union Steve Olive Deputy Mission Director for Somalia USAID Candace Buzzard Deputy Mission Director for Integration & Support Services USAID Josphat Wachira Regional Democracy & Governance/Conflict Advisor USAID Chihenyo Kang'ara Resilience & Adaptation Specialist USAID Isaac Thendiu Livestock Development Program Specialist USAID Daniel Cabet Deputy Director, FINANCIAL MANAGEMENT OFFICE (FMO USAID 92 Annex 9: Numerical Analysis of KII Data Whole of IGAD Issues: (648 Issues raised by 106 respondents) ISSUES RAISED: Issue raised by respondent regards predictibilty of donor or member state support as Strength or Weakness Issue raised by respondent regards IGAD Planning (including M&E) as Strength or Weakness Issue raised by respondent regards 'Empire Building' or 'Client Capture' as Strength or Weakness Issue raised by respondent regards potential (or past) IGAD capability as Strength or Weakness Issue raised by respondent regards level of understanding of IGAD constraints as Strength or Weakness Issue raised by respondent regards recognition of 'real world' constraints as Strength or Weakness Issue raised by respondent regards IGAD Human Resources as Strength or Weakness ALL ISSUES General Regarded as Weakness 394 101 74 10 33 111 301 69 Regarded as Strength 254 11 11 4 207 10 37 20 Sum 648 112 85 14 240 121 338 89 % S 39.2% 9.8% 12.9% 28.6% 86.3% 8.3% 10.9% 22.5% % W 60.8% 90.2% 87.1% 71.4% 13.8% 91.7% 89.1% 77.5% CSOs/NGOs CSO_S 32 0 4 0 29 3 8 2 CSO_W 85 14 16 1 6 35 74 7 Total 117 14 20 1 35 38 82 9 %S 27.4% 0.0% 20.0% 0.0% 82.9% 7.9% 9.8% 22.2% %W 72.6% 100.0% 80.0% 100.0% 17.1% 92.1% 90.2% 77.8% Member State Agencies GOV_S 76 1 5 1 70 3 13 2 GOV_W 75 14 16 4 5 22 54 11 Total 151 15 21 5 75 25 67 13 %S 50.3% 6.7% 23.8% 20.0% 93.3% 12.0% 19.4% 15.4% %W 49.7% 93.3% 76.2% 80.0% 6.7% 88.0% 80.6% 84.6% IGAD Agencies IGAD_S 94 11 6 2 59 2 8 14 IGAD_W 140 54 18 3 7 31 107 43 Total 234 65 24 5 66 33 115 57 %S 40.2% 16.9% 25.0% 40.0% 89.4% 6.1% 7.0% 24.6% %W 59.8% 83.1% 75.0% 60.0% 10.6% 93.9% 93.0% 75.4% Djibouti DJIB_S 56 5 2 1 35 1 3 11 DJIB_W 81 29 7 1 3 10 55 30 Total 137 34 9 2 38 11 58 41 %S 40.9% 14.7% 22.2% 50.0% 92.1% 9.1% 5.2% 26.8% %W 59.1% 85.3% 77.8% 50.0% 7.9% 90.9% 94.8% 73.2% Ethiopia ETH_S 60 5 5 0 47 3 12 4 ETH_W 73 28 16 3 9 21 55 10 Total 133 33 21 3 56 24 67 14 %S 45.1% 15.2% 23.8% 0.0% 83.9% 12.5% 17.9% 28.6% %W 54.9% 84.8% 76.2% 100.0% 16.1% 87.5% 82.1% 71.4% Kenya KEN_S 61 2 2 2 56 2 6 2 KEN_W 86 17 12 2 6 22 65 12 Total 147 19 14 4 62 24 71 14 %S 41.5% 10.5% 14.3% 50.0% 90.3% 8.3% 8.5% 14.3% %W 58.5% 89.5% 85.7% 50.0% 9.7% 91.7% 91.5% 85.7% South Sudan SS_S 39 1 2 1 32 3 8 3 SS_W 72 14 8 4 4 24 57 11 Total 111 15 10 5 36 27 65 14 %S 35.1% 6.7% 20.0% 20.0% 88.9% 11.1% 12.3% 21.4% %W 64.9% 93.3% 80.0% 80.0% 11.1% 88.9% 87.7% 78.6% Uganda UGA_S 38 0 25 37 1 8 0 UGA_W 82 13 62 11 34 69 6 Total 120 13 87 48 35 77 6 %S 31.7% 0.0% 28.7% 77.1% 2.9% 10.4% 0.0% %W 68.3% 100.0% 71.3% 22.9% 97.1% 89.6% 100.0% Triangulated by Organization Type* Triangulation by Member State * 93 CEWARN-Specific Data: (288 Issues raised by 106 respondents) ISSUES RAISED: Current functionality of CEWARN System regarded as Strength or Weakness Practical ability of CEWARN to manage rapid responses regarded as Strength or Weakness Viability of proposed 'new' CEWR system regarded as a Strength or Weakness CEWARN ability to liquidate/account for funds spent is regarded as a Strength or Weakness Current foci of CEWARN planning regarded as a Strength or Weakness Appropriateness of 'pre-existing' CEWR system regarded as a Strength or weakness ALL ISSUES General Regarded as Weakness 201 33 41 71 15 60 22 Regarded as Strength 87 4 15 8 1 18 49 Sum 288 37 56 79 16 78 71 % S 30.2% 10.8% 26.8% 10.1% 6.3% 23.1% 69.0% % W 69.8% 89.2% 73.2% 89.9% 93.8% 76.9% 31.0% CSOs/NGOs CSO_S 22 1 4 3 0 3 14 CSO_W 70 10 15 27 6 17 8 Total 92 11 19 30 6 20 22 %S 23.9% 9.1% 21.1% 10.0% 0.0% 15.0% 63.6% %W 76.1% 90.9% 78.9% 90.0% 100.0% 85.0% 36.4% Member State Agencies GOV_S 31 3 7 3 0 6 14 GOV_W 42 4 15 16 3 8 2 Total 73 7 22 19 3 14 16 %S 42.5% 42.9% 31.8% 15.8% 0.0% 42.9% 87.5% %W 57.5% 57.1% 68.2% 84.2% 100.0% 57.1% 12.5% IGAD Agencies IGAD_S 15 0 1 1 1 8 6 IGAD_W 41 1 1 12 5 26 1 Total 56 1 2 13 6 34 7 %S 26.8% 0.0% 50.0% 7.7% 16.7% 23.5% 85.7% %W 73.2% 100.0% 50.0% 92.3% 83.3% 76.5% 14.3% Djibouti DJIB_S 4 0 0 1 0 2 1 DJIB_W 6 0 0 0 0 6 0 Total 10 1 8 1 %S 40.0% 100.0% 25.0% 100.0% %W 60.0% 0.0% 75.0% 0.0% Ethiopia ETH_S 21 0 0 0 0 0 1 ETH_W 30 2 6 7 2 18 1 Total 51 2 6 7 2 18 2 %S 41.2% 0.0% 0.0% 0.0% 0.0% 0.0% 50.0% %W 58.8% 100.0% 100.0% 100.0% 100.0% 100.0% 50.0% Kenya KEN_S 13 1 2 2 0 1 9 KEN_W 38 10 10 11 4 7 8 Total 51 11 12 13 4 8 17 %S 25.5% 9.1% 16.7% 15.4% 0.0% 12.5% 52.9% %W 74.5% 90.9% 83.3% 84.6% 100.0% 87.5% 47.1% South Sudan SS_S 17 1 1 2 0 3 11 SS_W 49 7 12 13 3 19 5 Total 66 8 13 15 3 22 16 %S 25.8% 12.5% 7.7% 13.3% 0.0% 13.6% 68.8% %W 74.2% 87.5% 92.3% 86.7% 100.0% 86.4% 31.3% Uganda UGA_S 32 1 7 2 0 5 19 UGA_W 69 10 13 34 6 9 6 Total 101 11 20 36 6 14 25 %S 31.7% 9.1% 35.0% 5.6% 0.0% 35.7% 76.0% %W 68.3% 90.9% 65.0% 94.4% 100.0% 64.3% 24.0% Triangulated by Organization Type* Triangulation by Member State * 94 IDDRSI/PCU-Specific Data: 43 Issues raised by 106 respondents ICPAC-Specific Data: 58 Issues raised by 106 respondents ISSUES RAISED: IDDRSI coordination functions to date regarded as a Strength or Weakness Recruitment and holding of staff for USAID supported technical positions regarded as a Strength or Weakness Ability to absorb USAID funded positions (as permanent IGAD staff) regarded as a Strength or Weakness Clarity of IGAD role in Knowledge Management regarded as a Strength or Weakness ALL ISSUES General Regarded as Weakness 27 9 5 6 13 Regarded as Strength 16 14 5 1 3 Sum 43 23 10 7 16 % S 37.2% 60.9% 50.0% 14.3% 18.8% % W 62.8% 39.1% 50.0% 85.7% 81.3% ISSUES RAISED: ICPAC current regional and seasonal forecasting products regarded as Strength or Weakness Human Resource Capacity of ICPAC regarded as Strength or Weakness ALL ISSUES General Regarded as Weakness 22 21 13 Regarded as Strength 36 36 5 Sum 58 57 18 % S 62.1% 63.2% 27.8% % W 37.9% 36.8% 72.2% 95 Annex 10: Stakeholder Misconceptions Discussion of Stakeholder Perceptions and Misconceptions All sizable organizations develop bureaucratic processes to regulate their interaction with their operating environment. Generally, the more complex the operating environment, the more bureaucratic layers are needed to govern an organization’s activities. There does appear to be significant frustration among many stakeholders with IGAD’s systems (viz. IGAD Secretariat, the CEWARN Secretariat, ICPAC and the IDDRSI PCU) inability to operate in timely and effective enough ways to achieve many of their stated objectives. Commonly stated causes of frustration included:  The highly bureaucratic and multilayered administrative and decision-making processes of the IGAD organizations, and  The unwillingness of these IGAD organizations to push for progression of certain agendas or actions within member states. Given the consistency with which these issues were reported to the evaluation team by donor agencies, there can be little doubt that there is some substance to these complaints. While it is not the intent of this evaluation to make excuses for the IGAD agencies reviewed and to present a balanced position we must recognize that some causes of frustration with IGAD agencies can be attributed to incorrect assumptions that equate them with other implementing partners, such as international NGOs or contractors. It is important to address these potential misconceptions, because, if a clearer picture of the constraints under which IGAD operates is established, assistance to them can be more effectively targeted. Perceived Problem 1: Bureaucratic and multilayered administrative and decision-making processes IGAD and its agencies face undeniable problems with bureaucracy. Examples provided to the evaluation team included decisions or budget allocation/transfer/acquittal occurring so slowly that these activities effectively become irrelevant to their original purpose. While such extreme examples obviously require urgent attention, there will always remain a practical limit to streamlining IGAD processes. IGAD is a secretariat for political heads of member states. Under the agreement establishing IGAD, it is given no independent authority to formally act in a manner contrary to the wishes of even a single member state. The systems developed within IGAD organizations therefore inherently incorporate an exhaustive degree of accountability to their member states. Decisions must be checked (and often counter-checked) at numerous technical, administrative, and political levels, so that member states are assured that any actions or outcomes will not be detrimental to their national interests. While this is standard practice for any regional organization, especially aspiring regional economic communities, the political tensions between countries in the IGAD community exacerbate the issue. It is therefore necessary to accept that the level of bureaucracy IGAD needs internally, solely in order to exist, will always frustrate external stakeholders. If IGAD became significantly less bureaucratic, it could losing the trust of its member states and therefore become profoundly inoperative. It could be wise for external donors to adapt their assistance with recognition for these limitations, rather than expect them to change. This should not be regarded as a defeatist attitude, nor should it encourage disengagement from IGAD. What IGAD 96 can do, even within these limitations, includes many unique opportunities that likely could not be delivered by any other implementing partners or contractors, particularly initiatives related to the development of regional reference policies and convening of bilateral or regional fora and committees at multiple levels. Donors must also recognize that their engagement can have the potential to exacerbate problems of bureaucracy within recipient organizations. For example, if an organization with limited financial management capacity receives small tranches of funding from a wide range of donors and each donor has different procurement and accounting requirements, the organization’s financial system will be excessively stressed. This has been true for IGAD, who has approached donors for assistance in defining financial standards that will be universally acceptable. Donors have been reluctant to change their requirements, suggesting their limited ability to compromise their own bureaucratic constraints. Perceived Problem 2: The unwillingness of IGAD agencies to push for progress on certain agendas or actions within member states As stated above, even at the highest level, IGAD is merely a secretariat for political heads of member states. It is not, and cannot be, an independent or politically-neutral organization, as this would defeat its primary raison d’etre of representing the collective interests of the member states. This means that all IGAD agencies must be cautious when dealing with government bodies of member states; despite its mandates, IGAD staff are not formally recognized in member states’ government staffing structures and therefore have no direct authority over anyone within government bodies. Ongoing implementation of programs requires sustained good will that may be withdrawn at any time. This problem is best illustrated in the context of CEWARN. The CEWARN Secretariat has cooperatively negotiated implementation of shared standards and architecture for gathering certain conflict early warning data. It has also reached agreement that each member state will set up a (usually government-led) CEWERU to operationalize these structures. While the CEWARN Secretariat has been successful in promoting these standardized systems, issues and tensions have arisen when such systems attempt to transfer data directly from the point of collection to the CEWARN Secretariat, or when data requested by the CEWARN Secretariat are regarded by member states as having political sensitivities or national security implications. Representatives of national security agencies on CEWERU steering committees generally keep track of information being passed outside their borders, rather than contribute to it. In certain cases, member states views are wholly understandable and justifiable, particularly given tensions in the region. These constraints have resulted in a natural evolution of the direct operationalization of CEWARN systems and initiatives within member states. Generally, in over a decade of piloting such systems, successful direct involvement by the CEWARN Secretariat in member states has been limited to conflict early warning and responses addressing “community-to-community level conflicts that are either directly cross border in nature or have the potential to indirectly create ‘mirrored’ conflict over borders”. The reason direct involvement in data collection and responses for these types of conflict within member states appears to have been accepted is that it is in the interest of all countries to keep such conflicts in check. If left unchecked, minor cross-border conflicts can easily develop into serious and unintended bilateral problems, which may in-turn interfere with existing national security strategies. Despite inclusive mandates, CEWARN systems are subject to additional unspoken limits in the implementation of their systems. This said, the evaluation found that until mid-2014, the early warning and response systems that were developed within member states were largely consistent and effective and remained primarily based on the infrastructure originally promoted by the CEWARN Secretariat, even when systems were domesticated to include conflict types considered internally 97 sensitive to member states. (In such cases data collection/analysis/storage and associated responses did not include direct CEWARN Secretariat involvement). If the constraints in these member states can be expected to remain consistent over the next five or so years, the CEWARN Secretariat’s direct interventions may also remain constrained to addressing “community-to-community level conflicts that are either directly cross border in nature or have the potential to indirectly create conflict on two sides of a border.” This is not necessarily a bad thing. First, it can inform better focused and more effective short- to medium-term action planning. Second, addressing these specific types of conflict is extremely important to stability and associated prosperity in the region. While reduction in deaths is difficult to quantify, it is possible that hundreds to thousands of lives have already been saved by CEWARN Secretariat-led interventions. Note that other parts of IGAD have already demonstrated the ability to address higher level or more sensitive conflicts. A limited CEWARN Secretariat focus would therefore not define the total role of IGAD in regional conflicts. The fora and other diplomatic channels created by IGAD provide critical opportunities for formal and informal diplomacy between member states. Blocks of member states have already demonstrated that they are capable of using these mechanisms to counter unconstructive positions adopted by recalcitrant members. External stakeholders have the opportunity to participate in such mechanisms, using the already established “IGAD plus” groupings and fora. If IGAD seeks additional formal means of promoting certain reforms within member states, these will need to be largely based on incentive rather than punitive means. An example might be for IGAD to seek approval to formally include the explicit intent to merge with the East African Community (EAC) in its long term strategy. While it is recognized that this decision is well beyond the scope of this evaluation to recommend, it would provide IGAD an opportunity to promote the considerable economic benefits that such a merger would bring to its member states, while reminding them that the merger cannot occur until IGAD members match specific economic openness, political procedures, and other standards adopted by the EAC. If IGAD agencies and their funders were to take a more pragmatic approach to current constraint￾mapping, rather than base decisions on ambitious interpretations of long-term mandates, feasible results may be delivered more consistently and a great deal of wasted effort and frustration may be avoided. Another practical and often frustrating result of IGAD’s agencies’ inherent inability to directly implement or enforce implementation of activities in member states is an accountability gap. Once funding or assistance channeled through IGAD organizations is provided to member states, very little can be done to ensure effective monitoring or implementation. Member state agencies have no reason to fear IGAD criticism for inaction, but the IGAD organizations remain accountable to funders. Donors to IGAD may need to consider applying funding in a manner that allows member state agencies to be held more directly accountable for their results and approaches. The discussion of the two problems provided above is intended to help all stakeholders reexamine their assumptions about how best to provide assistance through IGAD. If these misconceptions do have currency, it is appropriate and important for all stakeholders to step back and consider what could be achieved if changes are adopted. 98 Annex 11: Lessons Learned Lessons Learned from the Evolution of the CEWARN System Introduction and Background In implementing the former conflict early warning and response systems, member states created their own national CEWERUs and utilized field monitors, local peace committees and national (but usually nongovernmental) agencies to compile and analyze data (such as national research institutes, NRIs), and national (but usually nongovernmental) agencies to administer external funding for relevant CSO projects (financial administration units). The capacity-building provided by the CEWARN Secretariat is widely regarded as a key factor in enabling national players to establish such regionally-standardized systems in their own countries. Other resources flowing directly from the CEWARN Secretariat were limited to support for establishing local situation rooms or longer term, response-related activities, such as peace dividend projects implemented through CSOs. Rapid response mechanisms proposed to be implemented through the CEWARN Secretariat were largely impractical and inoperative for that purpose.222 The lack of effective assistance from the CEWARN Secretariat in regard to rapid responses, left an obvious gap in the national systems. National and local rapid response mechanisms therefore evolved to fill this gap using locally-available resources. Under the systems that consistently evolved within member states, field monitors worked very closely with local peace committees; the ‘first use’ of any early warning information they received was consultation with the peace committees. The membership of these peace committees is generally diverse, including local CSOs and sub-national (district/provincial) government authorities. If a threat was decided to be significant and able to be addressed through local action, the peace committee would design and implement a response. In many cases, police and military intervention would be deemed necessary. If such agencies were not directly represented on local peace committees, the sub-national government authority member(s) would coordinate that part of the response. When such processes were functioning well, such local responses could be achieved on a same-day basis, if necessary. If a threat were deemed significant, but local resources were insufficient or higher-level approvals for action were required, the field monitor or local peace committee would elevate the issue to the national CEWERU. If the CEWERU felt a response was warranted and that it was capable of designing and implementing an effective response, it would do so. These responses again often involved police or military personnel, and could, in some cases, be delivered within a matter of days, if required. If the CEWERU considered it necessary to include regional components to a rapid response, the threat would be raised with the CEWARN Secretariat itself. If the regional assistance required was limited to convening or connecting relevant stakeholders across borders and appropriate CEWARN staff were in the office at the time, this could happen within the order of a week or so, and therefore still retain significant effectiveness. If the response required transferal of funding from the CEWARN Secretariat, this would (theoretically) mean that the response would have taken a matter of months. 223 This negated the CEWARN Secretariat’s ability to implement rapid responses. Note that 222 Finding B2. 223 Finding B4 99 in the later years of the ‘system’s operation, member state stakeholders could not recall a single case of a necessary rapid response having been first identified by the CEWARN Secretariat itself, and data fed into REPORTER by NRIs reportedly did not return in any analyzed form to member states for, again, a matter of months.224 If sensitive internal or bilateral conflicts were involved, the CEWARN Secretariat could elevate the issue to the Secretariat level of IGAD. This was both appropriate and necessary, because the political capacity of the CEWARN Secretariat is limited to discussions at regional committees of permanent secretaries. The IGAD Secretariat encompasses committees extending to both ministerial and heads of state levels. In all cases, field monitors collected the required early warning data and sent it up appropriate channels. It first went to the NRI, which conducted preliminary analysis of data and loaded it into the REPORTER system for transmission to the CEWARN Secretariat. When individual field monitors were not able to see broader patterns, the NRI had the prerogative of engaging directly with the national CEWERU for development of an appropriate response. The application of national police and military resources to responses is a largely hidden, but very considerable national contribution made to the system, 225 and represents a considerable long term political and financial commitment by many member states, who appear to have accepted these resourcing requirements without complaint or question, and tend to regard them as in their national interest. Hence, there is no indication that this commitment will wane in the foreseeable future. That this has occurred in member states that have been most reluctant to pay their annual commitments to IGAD, is of particular interest. It seems to suggest that these states favor a “pay as you go” approach to funding IGAD initiatives, over paying annual dues. In most cases, systems remained focused on the CEWARN identified border clusters. However, in special cases such as Kenya, not only did a system such as this function effectively in clusters, but the national CEWERU took the initiative to extend these standardized approaches (primarily using peace monitors and local peace committees), to respond to election-related violence. Eventually the Kenyan National CEWERU covered virtually the entire country and had dedicated very significant resources to its establishment and operation. This extension was, however, regarded as an internal Kenyan concern, so formal information flows for electoral violence early warning systems stopped at the CEWERU level. Potential Lessons While the complete domestication of the system may concern some stakeholders, a clear lesson that could be learned from the evolution of the pre-existing system across all member states, is that there will always be limits that national governments will place on information flows directed outside their borders. The fact that the regionally-standardized system is used for internal purposes is the next best outcome, because it means that if such controls are eventually relaxed in a particular country, the national systems can be immediately subsumed into the compatible regional systems. Domestication of systems contributes to sustainability through both transferal of resourcing and opportunities for expansion. It may be argued that the CEWARN Secretariat’s reaction to the nationally-controlled expansion in 224 Secondary Data Reference 225 Secondary Data Reference 100 Kenya was one of concern (possibly due to Kenya’s ability to attract separate funding from external donors). This conjecture is supported by the fact that other CEWERUs report a subsequent urgent push from the CEWARN Secretariat to expand into thematic areas such as election violence, with the clear intent that formal information flows from such expansion would reach the CEWARN Secretariat. While the intent of this push may be laudable, it is ill-advised as it ignores real world constraints and sensitivities. Working more patiently to capitalize on gains to date, will likely produce better outcomes sooner, and prevent future hiatuses in operations. Any attempts by the CEWARN Secretariat to require access to nationally sensitive data could easily result in abandonment of the regionally-standardized early warning and response frameworks for some forms of conflict. 226 This would degrade both informal information gathering opportunities and future opportunities for incorporating other types of conflict into the regional framework on a country-by-country basis, as it becomes permissible. Pushing too hard and too quickly for expansion of the types of information gathered by the CEWARN Secretariat itself could threaten the long-term stainability of the regionally-standardized frameworks needed to gather such data. The situation presented above is an amalgam of varying levels of progress across all member states. While this parallel evolution of conflict early warning and response systems may not have been in strict accordance with the CEWARN Secretariat policy or wishes, it was primarily based on the original, regionally-standardized architecture promoted by CEWARN. This level of success must be regarded as a significant achievement in itself and the fact that many countries (including CSO membership of the CEWERUs) took it upon themselves to overcome initial design flaws and develop the system into something more effective should be welcomed. The additional fact that evolution of the initial system resulted in similar outcomes in multiple countries provides strong evidence of the appropriateness of this end-product system. Indeed, many CSO respondents saw a key role of their involvement in the delivery of the standardized systems to one of overseeing a trial or pilot system, with progressive incorporation of improvements a natural occurrence. Acceptance of the results of this trial and formal recognition by the CEWARN Secretariat of appropriate innovations made to their original system over the past decade should be a priority for the agency, both in terms of gaining credit for their regional achievements and ensuring long-term sustainability of a standardized regional conflict early warning and response system. Another clear lesson that can be taken from the system’s evolution to date is the need to keep data gathering, analysis, and response systems simple enough to be locally relevant and applicable. For example, if a field monitor cannot immediately understand or recognize the significance of the data they collect, they will lose the important ability to initiate action at the local level. If this occurs, they may resort to independently collecting their own forms of data to inform the need for a response, which would demonstrate the irrelevancy of the data required by the CEWARN Secretariat. This need to retain a focus on what works is also an important sustainability issue, as the CEWARN Secretariat is currently trying to progress a proposal for a new system that significantly expands the types and amount of early warning data collected, by separating it into five distinct dimensions for analysis by five NRIs.227 This change is concerning, as it may render analysis by laypersons such as field monitors, local peace committee members, and (some) CEWERU members, who should be regarded as the primary users of the information. The academic merit of collecting more complex and comprehensive data types is recognized, as is the need for undertaking some overall trend analyses. However, the CEWARN Secretariat must remain aware that operational systems should not mistake 226 Note this does not exclude more sensitive advocacy approaches to data sharing. See Finding C4 as well. 227 Viz. From data 101 international researchers at overseas workshops or conferences as key clients.228 Implementing and maintaining a regional conflict early warning and response system is far from an academic exercise, especially in a region where real world constraints, including bilateral tensions, remain relatively high. In such scenarios, academic pursuits should only be pursued if they are fully aligned to functional outcomes (e.g., trend analysis or good regional practice research). This can be linked back to earlier discussions relating to the advisability of core business planning within all IGAD agencies. 229 Not all end-points reached in the evolution of the original CEWARN systems at country level have been ideal. Considerable areas remain where improvements could be made to render them both more effective and sustainable. For example, because support from the Rapid Response Fund was only reliable for longer-term, response-related initiatives, such as peace dividend type projects, 230 resourcing of CSO responses has been limited. 231 This has impeded the process of further defining and recognizing appropriate roles for CSOs in immediate conflict responses (beyond basic dialogue facilitation). Because shortfalls in CSO involvement have needed to be resolved by police or military actors, CSO roles have been downplayed while police and military roles have been exaggerated in immediate responses. While involvement of police and military forces in some rapid responses is considered essential by all stakeholders, there are also fears that excesses may occur when responses are left to these agencies alone.232 The CEWARN Secretariat is in a position to address both these issues to some extent, including support for and facilitation of more effective approaches to resourcing CSO participation in rapid responses and developing guidelines or a code of conduct covering roles of all players in rapid responses conducted under any regional CEWARN system. The CEWARN Secretariat is in a unique position to negotiate and obtain member state sign-off on such guidelines or codes of practice.233 While such documents may not have immediate effects, their long￾term role as a reference policy for the region may be considerable. Lesson 1 in this annex provides a more complete list of issues raised by CSOs related to the proposed new system. Key Lessons Key Lesson 1: The pre-existing conflict early warning and response system, as evolved by member states has been demonstrated to be effective, reasonably efficient, already inclusive of a degree of CSO involvement and design input, as well as suited to substantive expansion of CSO involvement and input. 234 This lesson is clearly at odds with extant CEWARN Secretariat policy and nascent actions taken to change to a new system, which as of the evaluation period has neither been tested nor effectively implemented. This contradiction has caused semantic issues within this report, as the authors have found it difficult to define which to call the current system. In fact, the easiest solution to this problem has been to simply acknowledge that since approximately June 2014, the CEWARN Secretariat has effectively had no conflict early warning or response system in place. In short, there is no current system. 228 This does not mean that international research agencies or conferences should not have access to any information generated, merely that information collection should not be designed with such lower priority clients’ needs in mind. 229 Finding A4 230 As it could simply not be accessed quickly or reliably enough to mount sufficiently rapid response. 231 National governments are, arguably, wisely wary of funding CSO responses because CSOs are not reliably non-partisan. 232 Historical reference to disarmament project examples UGA-Kenya 233 Using elevation of the matter to IGAD Secretariat level, as necessary. 234 And characterized by use of field monitors, local peace committees, and localized response mechanisms. 102 This collapse of the CEWARN Secretariat controlled elements of the system is not just limited to the abrupt termination of field monitors, but also includes discontinued operation of NRIs and reduced functioning of local peace committees, which were heavily dependent on the field monitors for information and guidance. Where CEWERUs have been able, they have preserved functional remnants of the pre-existing system. Rather than critique the detail of this proposed new system, it is equally effective, and far more efficient, to critique the flawed premises upon which the changes appear to be based. The change also seems to have had external donor support. These donors were reluctant to renew funding for the field monitors in particular, as they felt the field monitors failed to encourage substantive involvement of CSOs. They also believed using CSOs for data gathering would be a less expensive, and therefore more sustainable, option for data collection. However, the information obtained from respondents to this evaluation, particularly those respondents currently working for CSOs in peace and conflict fields, strongly points to both of these assumptions being incorrect. Key points raised by such respondents include:  Most of member states’ conflict early warning systems were designed and initiated by the CSO/NGO community, and were later adopted by CEWARN.  Many ex-field monitors were already employees of CSOs and worked under the auspices of those CSOs.235  In most member states, CSOs have been largely responsible for the testing and ongoing tweaking of the pre-existing system.  During the period under evaluation, the proposed new system had not been tested.  Most local CSOs generally have insufficient capacity to provide high quality early warning data. While some may have their own systems, these are functionally-based and do not match the greatly elaborated data collection required under the proposed new system.  Local CSOs cannot be expected to substantially increase the scope of their operations without clear and substantial incentives to do so.236  Incentivising or building the capacity of individuals within CSOs will always remain less costly than incentivising or building the capacity of entire organizations.  Any assumption that CSOs are consistently neutral and non-partisan is not reliable (e.g., some CSOs are political parties and many are politically affiliated).  Many CSOs follow the best funding opportunities, so long-term minor relationships with a range of CSOs in specific locations may not be sustainable.237  Collapse of the pre-existing system has damaged the trust and confidence that local communities have in the early warning and response processes in which they participated and needs to be redressed urgently, given that this trust and confidence was one of the major drivers of success in conflict mitigation.238  Some CSOs are now suspicious that the change to a new system reflects collusion between the CEWARN Secretariat and member states governments to push out existing experienced CSOs and replace them with tame CSOs.239 235 e.g. in Uganda five of the seven field monitors supported by CEWARN were embedded in CSOs. 236 Some CSOs appear to have been told that the reward for participating in the new system would be the ‘association’ with CEWARN (including potential access to Rapid Response Funding). Given recent CSO experience with CEWARN funding mechanisms for field monitors, NRIs, etc., this suggestion may have been counter-productive. 237 Whereas capacity built in an individual resident in a location can be moved from one CSO to another. 238 And took over a decade to establish. 239 The fact that the evaluation team did not find any evidence to back this suspicion needs to be noted. However, this fact is also largely irrelevant, as the perception clearly exists amongst CSOs. 103  Some CSOs involved in the development of the CEWARN 2012-2019 Strategic Plan believe that changing to this new system contradicts what was agreed to in that document, as no mention was made of the intention to change at that time.240 The CEWARN Secretariat can be held partially responsible for the current collapse of systems for not openly defending or revealing the extent to which their pre-existing systems depended on CSO involvement. However, donors could have examined the pre-existing systems more closely before judging that they had inadequate CSO involvement. This lesson does not discount flaws that remained in the pre-existing system. Suggestions for further improvements were forthcoming from many CSO respondents. A particularly valuable example was that the CEWARN Secretariat more clearly define CSO roles in responses to potential or existing conflicts. This should include but not be limited to existing roles, such as facilitation of dialogue. This step would further legitimize CSO involvement in such responses and promote options other than existing police or military action. Another critical issue remains the sustainability of CEWARN’s ability to continue to obtain data, given funding constraints. Respondents point out that the means of collecting data to inform responses to conflict have grown considerably over the last decade. This is correct, but a number of constraints identified in the evolution of the pre-existing systems temper this view: 1. The first constraint of note is that there is a significant difference between potential and acceptable means of gathering data on conflicts. Member states are not willing to give blanket approval for the collection of potentially sensitive data regarding conflict to be delivered to the CEWARN Secretariat in any manner that bypasses their ability to scrutinize it first. This is understandable both from a national security perspective, and from the need to localize responses, if they to ensure they are to be rapid enough to be effective (see Lesson 3). Hence, ideas such as crowd sourcing of data directly by the CEWARN Secretariat using SMS networks may not be feasible in some countries.241 Any proposed system that assumes open sharing of any expanded form of data collected will not likely be successful in the foreseeable future. 242 2. Data collection for any specific purpose requires resources, which has flow-on implications. For example, the need/desire by national meteorological agencies to obtain a return on the considerable investments made in collecting raw weather and climate data is the primary reason ICPAC cannot share such raw data to potential external clients.243 Given that the CEWARN Secretariat’s proposed system requires even greater breakdowns in data collected (to investigate five separated dimensions of conflict), it is unlikely that any CSO is already collecting information in precisely the form the CEWARN Secretariat would require. Even if they were, CSOs may have some proprietorial views on its ownership, given the effort they have gone to obtain it. In addition, the proposed system implies that the CEWARN Secretariat may expect the geographic expansion of existing data gathering conducted by CSOS into locations of the CEWARN Secretariat’s choosing. Any increase in expense or 240 And a close reading of the Strategy agrees with this. 241 Certain member state officials suggested that it would be illegal for any telecommunications providers to participate in such an organized intelligence gathering system and that their national telecommunications regulator would be instructed to prevent it happening. 242 Note that once governments allow movement of data to unsecure or unregulated databases, they must assume open sharing will take place, and the more sensitive the data, the more regulation and security such databases would need to demonstrate to negate this view. Imposing military or police level regulation on CEWARN data bases is not a practical option for an organization that seeks substantial CSO involvement. 243 Such restrictions on raw data sharing are completely consistent with WMO guidance. 104 personal risks for CSO staff must considered in the development of the new system. Given that both the pre-existing or proposed system require funding, “who should pay,” is a key question. From a sustainability perspective, it may appear that the best option would be for member states to assume this responsibility. This has happened in relation to payment of field monitors in some member states that have extended systems to cover other forms of internal conflict, most notably, election violence in Kenya. However, Kenya has a comparatively strong record of CSO involvement and therefore retained CSO-reliant models in expanding systems to cover election violence. Not all member states have such strong records of CSO involvement,244 so requiring them to take over funding of data gathering may result in reduced CSO involvement in the short term. If the objective is to retain or expand CSO involvement, both the CEWARN Secretariat and donors may need to acknowledge that for the foreseeable future, this may require external funding. Key Lesson 2. Responses to early warnings of conflict should be generated where the lowest, contextual opportunity exists for doing so appropriately and effectively. If an appropriate and effective response is possible within an affected community, any response system should actively encourage such action. It should not attempt to shunt responsibility for action to a higher level than absolutely necessary. The main reasons for this position include:  Local approaches (within appropriate guidelines)245 are most likely to be fastest and the most contextually informed,246and therefore the most effective and durable;  Local approaches inherently encourage self-sufficiency and are therefore the most likely to lead to sustainable systems and solutions; and  Encouraging the lowest level action as a first resort avoids any unnecessary escalation or over-reaction. This does not imply that responsibility for responses should never be pushed to higher levels. Rather, that responsibility for action should go only as high as is necessary to effectively address the problem. This clearly implies the need to train staff at the lowest levels of systems (e.g., field monitors and peace committee members) to recognise the most appropriate level at which to raise a response, and how to design an appropriate response, if a local response is feasible. Encouragement of the most local effective responses and explicit recognition of their importance is not overly apparent in the proposed new CEWARN Secretariat system. While information gathered should, where sanctioned by member states, eventually reach the CEWARN Secretariat for analysis of overall trends, expectations of rapid analyses and responses by the CEWARN Secretariat itself appear to be unachievable.247 However, despite the practical exclusion of rapid analysis and response capability, the CEWARN Secretariat retains some vital functions. Its role in convening cross-border meetings and other contact has been critical to initiating and harmonizing member state efforts across borders. Much of the lasting success national CEWERUs have had in preventing reoccurrence of conflict within relevant clusters is attributable to the role of the CEWARN Secretariat.248 The CEWARN Secretariat also provides a critical link to the higher level political capacity of the IGAD 244 e.g. in Ethiopia only two CSO are permitted, by law, to operate in the peace and conflict field, 245 See recommendation 6. 246 Respondents cited examples where responses were able to be organized at community level within the same day the conflict warning was received (and at CEWERU level within a few days). 247 See lesson 3. 248 Alignment of cross-border disarmament efforts are a good example, because disarming only one side of the border can increase conflict. 105 Secretariat. When appropriate responses have required high level regional diplomacy, matters have been formally or informally referred by the CEWARN Secretariat to the IGAD Secretariat, which has political capacity and access of up to heads of state level.249 Key Lesson 3. Truly rapid responses cannot occur if responsibility to identify, plan, approve or fund such responses has to be passed up to the CEWARN Secretariat level. While not all appropriate responses to potential or actual conflict can or should be immediate (see lessons 1 and 2), rapid or time-sensitive responses are often critically important to preventing occurrence or escalation of conflicts. Informal canvassing of respondents to this evaluation has suggested that, to be effective, rapid responses must be measured in the order of hours or days, rather than weeks or months. The fastest respondent-reported case of directly accessing Rapid Response Fund resources was in the order of months. The CEWARN Secretariat responded to this problem by placing a standing fund of approximately USD10, 000 with each CEWERU, but replenishment and accountability for liquidation of such standing funds continued to be an ongoing problem. Availability of pooled funding at the CEWARN Secretariat level may remain sensible for longer-term responses to conflict, such as peace dividend projects. the CEWARN Secretariat retains the (albeit largely unrealised), potential to use such projects as research vehicles into good practices that can then be disseminated to sufficiently similar contexts in the region. However, it does not seem that the CEWARN Secretariat is able to take responsibility for truly rapid data analysis and responses at this time. Even if approval of such action can bypass the normal sittings of CEWARN’s Rapid Respond F und Steering Committee (such as by utilization of virtual meetings), the CEWARN Secretariat has limited human capacity to quickly assess situations and approve action. This limitation in staffing is not only technical, but also numerical. If staff are travelling on other duties, or are simply on leave, alerts and proposals can go unattended for considerable periods. The good news is that many member states appear willing to fund responses requested by field monitors, local peace committees, or their own CEWERU at short notice. These responses and contributions often go unreported at the CEWARN Secretariat level as they use national police or military agencies. While this is universally regarded as a necessary option in some conflict responses, national forces should not be the only available option. The inability of CSOs to rapidly access contingency-based funding for conflict responses elicits a bias toward use of such forces. Key Lesson 4. The CEWARN Secretariat does not have the financial, human, technical, or political capacity to do everything connected to conflict. As discussed in Annex 1, all organizations have their limitations. The CEWARN Secretariat is no exception to this rule. Despite an expansive and inclusive mandate on paper, experience in implementation over the past decade has demonstrated practical and political constraints that limit the secretariat’s ability to fulfil its mandates. Though where member state governments have signed the original mandate, changes in governments lead to changes in preferences and sensitivities regarding conflict related issues. In some cases, this had led to rapid expansion of CEWERU responsibilities in ways that do not directly involve the CEWARN Secretariat (e.g. the Kenyan CEWERUs extensive role in election-related violence). In other cases, it has led to CEWERUs 249 The South Sudan and Somali conflicts have largely been handled this way. 106 resisting involvement in nationally sensitive issues, or being directed to leave such issues to national security agencies. Somewhat ironically, it has been CEWERUs expansion and domestication of the standardized systems to encompass broader issues that may have caused the CEWARN Secretariat the most concern; other CEWERUs reported that the CEWARN Secretariat requested that they also expand the types of conflicts and areas covered.250 This response may have been the CEWERU Secretariat’s attempt to disseminate good practice, or it may be that the secretariat does not want countries developing systems that the Secretariat does not directly control; if some countries were to do this and not others, they could potentially attract separate funding from external donors. The irony is that such cases of system domestication represent an almost ideal outcome for conflict early warning and response systems in the region. It is largely unavoidable that member states will place limits on who can receive or respond to certain types of conflict related information from within their borders. Therefore extending the regionally-standardised systems, structures, and mechanisms developed with the CEWARN Secretariat’s assistance to cover sensitive conflict-related issues internally, as was done in Kenya, is an appropriate adaptation. Should sensitivities decrease, this standardized nature of the national systems provides potential for future CEWARN Secretariat engagement in these issues. The CEWARN Secretariat’s role as a service provider to member states for capacity-building of CEWERU staff in the operation of such systems also means it may retain strong informal involvement in internal systems. This will allow the CEWARN Secretariat to remain informally informed across a wider range of conflict issues than is formally possible. If necessary, such informal knowledge can be raised at higher levels within IGAD, through either informal alerts or other prompts for diplomatic action. Simply encouraging CEWERUs to “keep-up with their neighbors,” may alienate less developed member states. Identifying specific successes and constraints in each case may help inform which types of conflict or locations each state can become directly involved in, and where it may need to accept that facilitating domestication of systems and structures for sensitive issues is the best option. The challenges and successes demonstrated by the decade-long trial of the pre-existing conflict early warning and response system should guide the CEWARN Secretariat’s implementation of the 2012- 2109 strategy. It would appear that the CEWARN Secretariat has had most success in its involvement in community-level cross-border conflicts. “Community-level conflicts that involve the crossing of national boundaries, or conflict that involve ethnic or cultural groups have the ability to generate mirrored conflict on other sides of national boundaries.” While this definition of CEWARN’s competitive advantage may need further refinement, it should be noted that this does not limit them to their traditional focus on pastoral conflict. Even this definition would allow their direct involvement in local boundary disputes, land ownership disputes, issues regarding cross border migration or disarmament, or disputes arising around traditional practices such as female genital mutilation. For greatest medium-term effectiveness, it may be appropriate for the CEWARN Secretariat to consolidate its role to these specific forms of conflict. In particular, it could focus on trying to bring all member states to a more equal level of capacity for addressing these forms of conflict. In doing so, it will ensure that the regionally-standardized conflict early warning and response systems are adopted 250 Despite no functional CEWARN system being available to expand. 107 and implemented as well as possible. Facilitation the domestication of these systems to address more sensitive forms of conflict can also be considered. The ongoing gaps in capacity within the CEWARN Secretariat should be taken into account when considering any other roles in the medium-term. While the CEWARN Secretariat may have the ability to increase some of its capacity with additional funding, some is effectively fixed. For example, while technical capacity might be temporarily increased with addition of new project-based staff, political capacity is limited to the committee of permanent secretaries. This level of political capacity could only change if CEWARN committee membership was elevated to duplicate ministerial and heads of state committees already convened at the IGAD Secretariat. Attempting to replicate this type of political capacity at CEWARN would be redundant, confusing, and would further promote problems of empire building within IGAD. Highly technical or sensitive areas of conflict, such as countering terrorism or transnational crime, should remain the responsibility of specialized (police or military) forums within IGAD. Attempts to replicate or move such responsibilities to CEWARN are likely to be frustrated by real world constraints. If CEWARN were to move into intelligence gathering and responses in relation to terrorism or trans-border crime (beyond incidental involvement through its general conflict and early warning activities), this would require the CEWARN Secretariat to match the technical capacities of national police and military forces and/or multilateral agencies such as Interpol. As experience to date has shown, member states are, understandably, not always willing to override national security concerns or protocols in relation to information gathering and dissemination, especially when data security on intermediate databases cannot be overseen or guaranteed. It would also put the CEWARN Secretariat in a difficult position when internal member state politics blur the delineation of terrorists and criminals.251 The CEWARN Secretariat and donors should be cognizant that CEWARN’s written mandate is, in reality, a normative and aspirational statement, rather than a true reflection of its immediate capabilities. However, even within these limitations, CEWARN has strong potential to implement and facilitate regionally standardized conflict early warning and response systems. Key Lesson 5: Functional basic systems are far preferable and sustainable than dysfunctional impressive systems. Data gathering and analysis and response systems must be kept simple enough to be locally relevant and applicable. If field monitors cannot immediately recognize the significance of the data they collect, they will lose the ability to initiate action. If this occurs, they may resort to independently collecting their own forms of data to inform the need for a response. The CEWARN Secretariat is currently trying to implement a new system that significantly expands the types and amount of early warning data collected, by separating it into five distinct dimensions, 252 and having five NRIs analyze each dimension separately. This change is concerning, as it may complicate the data to such an extent that laypersons such as field monitors, local peace committee members, and some CEWERU members, who should be regarded as the primary users of the information, are unable to analyze it. The academic merit of complex and comprehensive data sets is recognized, as is the need for undertaking some overall trend analyses, but the CEWARN Secretariat must be careful not to mistake researchers at international workshops or conferences as key 251 Similar to that often faced by organizations such as Interpol, for example, see: https://www.hrw.org/news/2015/06/24/germany-investigate-detention-egyptian-journalist 252 Viz. From data 108 clients.253 Implementing and maintaining a regional conflict early warning and response system is far from an academic exercise, especially in a region where real world constraints, including bilateral tensions, remain relatively high. In such scenarios, academic pursuits should only be pursued if they are fully aligned to functional outcomes (e.g., trend analysis or good regional practice research), otherwise they may deter from more important work. This can be linked back to earlier discussions relating to the advisability of core business planning within all IGAD agencies.254 253 This does not mean that international research agencies or conferences should not have access to any information generated, merely that information collection should not be designed with such lower priority clients’ needs in mind. 254 Finding A4. 109 Annex 12: Recommendations for Revised CEWARN System Suggestions for CEWARN System Revisions Funding of operational aspects of the CEWARN system has always been primarily dependent on external donors. These donors are often reluctant to support regional or local systems in the long term, but in this case there appears to be no alternative. In recognition of the reservations expressed by donors, any revised systems must address concerns that funds are being used effectively and efficiently. In addition to lessons learned from the pre￾existing system, these suggestions incorporate input from financial controllers of USAID. It is also recognized that these suggested changes remain ambitious. A staged approach to implementation is therefore recommended and should be clearly reflected in CEWARN Secretariat work planning. It should be also be noted that the evaluation team does not provide these revisions as an attempt to create an ideal system. They are presented merely as the most appropriate compromises at this point in time. Neither do these suggestions attempt to cover all CEWARN functions. Some suggestions simply reconfirm existing functions or conditions. Key components and conditions: 1. Field monitors Revised conditions/roles:  Other than in locations where no relevant CSOs are available, field monitors will be selected from CSO staff;  Field monitors’ capacity will be built for a train-the-trainer role; they will be required to pass on information gathering and other conflict related skills throughout their CSO and other relevant CSOs that may be ‘networked’ in their area;  Field monitors in official CEWARN clusters will be funded through the CEWARN Secretariat (with donor assistance for the short to medium term);  Field monitors will be responsible for informing or coordinating community-level responses by other local actors, but only if a response at this level is feasible and advisable;  Field monitors’ capacity will be built to inform or coordinate community-level responses, and recognize whether a local response is feasible and advisable; and  Field monitors will continue to collect (revised) early warning data, either personally or through their expanded CSO networks and will: o Use it to inform or coordinate community-level responses, only if a response at this level is feasible and advisable; and o Supply these data to NRIs. 2. Peace Committees Revised conditions/roles:  Local (or sub-national) Peace Committees will continue to work closely with field monitors, including in initiating or coordinating community-level responses, but only if a response at this level is feasible and advisable;  The capacity of local (or sub-national) Peace Committees will be built to initiate or coordinate community-level responses and recognize whether a response at this level is feasible and advisable; 110  Peace committees will continue to be regarded as CSOs in relation to accessing CEWARN Secretariat funds and must comply with FAU requirements and assistance in accounting for related expenditures; and  Peace committees will continue to play a strong role in dialogue, and local solutions such as peace pacts will now be formally recognized as valid achievements of Peace Committees. 2. CSOs Revised conditions/roles:  CSO membership in CEWERU Steering Committees must continue and will become a fixed requirement in line with Articles 9, 10, 11, and 12 of the CEWARN Protocol, such that any CEWERU that loses CSO membership shall not access donor funds, either through the CEWARN Secretariat or directly, until this is corrected;  CSOs, particularly those hosting field monitors, must assist the CEWARN Secretariat to define the full range of their appropriate roles in conflict early warning and response; and  CSOs receiving Peace Research or Rapid Response funding must accept and comply with the assistance of the FAU in relation to accounting for expenditures. 3. Financial Administration Unit (FAU) Revised conditions/roles:  There will continue to be only one FAU per member state, guided by the Rapid Response Fund Framework;  The FAU’s formal responsibility will be extended to provide direct assistance to CSOs fully to account for expenditure of CEWARN Secretariat funds;  The FAU’s service fee will be increased to match these new responsibilities;  To prevent conflicts of interest, the FAU will not be permitted to access funds of CEWARN Secretariat to delivering new projects (FAUs will be permitted to complete projects funded by the secretariat established under the pre-existing system);  The FAU will be the repository for CEWARN Secretariat funds, and will only be permitted to release these funds to CSOs;  The FAU will release funds for “peace research” (formerly “peace dividend”) project proposals only on the authority of the CEWARN Secretariat;  The FAU will release funds for rapid response proposals on the authority of the national CEWERU or the CEWARN Secretariat; and  The FAU will accounts for all expenditures (e.g., field monitor payments, peace research projects and rapid responses), to both the CEWARN Secretariat and their CEWERU on a quarterly basis. 3. National Research Institute (NRI) Revised conditions/roles:  There will continue to be only one NRI per member state;  The NRI will continue to collect selected conflict early warning information from field monitors and other sources;  The NRI will continue to compile and analyze this information and pass the raw data and analyses to the CEWERU and the CEWARN Secretariat;  If an NRI lacks expertise in an important analytic field, it will be required to recruit or co￾opt such expertise to form part of its team of expert analysts;  The NRI’s formal responsibility will be extended to include monitoring and evaluation of: o Field monitor programs; 111 o CSO’s responses using CEWARN Secretariat funding; o CSO’s peace research projects using CEWARN Secretariat funding;  The NRI’s service fee will be increased to match these new responsibilities;  To prevent conflicts of interest, the NRI will not be permitted to access CEWARN Secretariat funds for delivering new responses or projects (NRIs will be permitted to complete projects established under the pre-existing system);  The NRI will be required to analyze monitoring and evaluation information and provide any necessary corrective or improvement advice to implementers as soon as practical; and  The NRI will compile the monitoring and evaluation information gathered, as well as any advice provided to project implementers, and provide this to their CEWERU and the CEWARN Secretariat on a quarterly basis. CEWERUs Revised conditions/roles:  CEWERUs will continue to respond to information provided to it from NRIs (or directly from field monitors or peace committees) and initiate or coordinate national responses, where needed;  CEWERUs’ capacity will be built to initiate or coordinate community-level, sub-national, or local level responses and recognize at what level a response is most appropriate;  If a national response is deemed incomplete or inadequate, the CEWERU will seek assistance from the CEWARN Secretariat in mounting a broader, multi-state response;  In designing responses, CEWERUs will specifically consider expanded roles for CSO involvement (provided these roles do not place CSO staff or community members in additional danger), and will facilitate CSOs’ access to rapid response funds;  CEWERUs will have conditional responsibility for approving funding to CSO rapid response proposals (dependent upon consistent delivery of quarterly financial reports to the CEWARN Secretariat by the FAU);  CEWERUs will be responsible for providing the details of all responses implemented using CEWARN funds to the secretariat on a quarterly basis;  CEWERUs will be required to record attendance lists and minutes of all formal meetings they convene and forward these to the CEWARN Secretariat on a quarterly basis;  CSO membership in CEWERU Steering Committees must continue and become a fixed requirement in line with provisions of the CEWARN protocol, such that any CEWERU that loses CSO membership shall not access donor funds (either through the CEWARN Secretariat or directly) until this is corrected; and  CEWERUs will be responsible for recording and validating the approximate cost of member state agency contributions* (financial or in-kind) to the conflict early warning and response system and providing this information to the CEWARN Secretariat on a quarterly basis (*including police or military agencies). The CEWARN Secretariat Revised conditions/roles:  The CEWARN Secretariat will review all early warning data gathered by the system to ensure it is focused on best informing local, national and regional responses.  In expanding the CEWARN system, a focus on “community-to-community level conflicts that are either directly cross border in nature, or have the potential to indirectly create ‘mirrored’ conflict over borders” should be maintained in the medium term.  When deemed necessary (on the basis of a CEWERU request or formal/informal information gathered), the CEWARN Secretariat will be responsible for initiating or 112 coordinating cross-border or multi-member state responses. In extreme cases this may mean referring the issue to the IGAD secretariat for higher level (Minister or Heads of State) action.  The CEWARN Secretariat will review all early warning data its system gathers, and the means used to gather it, to ensure its collection generates the least risk to data gatherers (including CSO staff).  The CEWARN Secretariat will be responsible for approving funding to peace research (formerly peace dividend) proposals.  Peace research (formerly peace dividend) proposals will only be accepted or approved if they are received from CSOs or NGOs (other than FAUs or NRIs). These may include CSOs or NGOs servicing more than one member state.  The CEWARN Secretariat will explicitly consider the value of each peace research proposal as a potential pilot of an approach that may be extended to similar contexts within the region  The CEWARN Secretariat will collate and summarize quarterly reporting from all FAUs and forward the summary and compiled individual reports to relevant stakeholders, including donors.  The CEWARN Secretariat will retain funds for peace research projects until they are approved for implementation.  The CEWARN Secretariat (with assistance of donors) will determine an annual allocation of rapid response funds for each member state and disburse this money in equal portions on a quarterly basis. (Note: allocations to each member state may differ and should be based on an annual assessment of likely need. CEWERUs may make exceptions for earlier than scheduled disbursements based on immediate need.)  The CEWARN Secretariat will prepare, and progress to member state ratification, a code of conduct or set of detailed guidelines specifying the roles and desirable limitations of all players (armed or unarmed) in responses implemented under the CEWARN conflict early warning and response system.  The CEWARN Secretariat will design a revised monitoring and evaluation system (separate to the conflict early warning and response system) for implementation by NRIs, and will compile and summarize the information received from NRIs on a quarterly basis, for release to appropriate stakeholders.  The CEWARN Secretariat will continue to review the early warning data supplied through the system and release alerts or other information to appropriate stakeholders on an as￾needed basis. It will also compile and summarize a trend analysis every quarter and provide it to appropriate stakeholders.  The CEWARN Secretariat will continue to provide capacity building and other support to CEWERUs, field monitors and peace committees, but expanded in scope to match their revised formal roles. External Donors External donors should consider providing funding or other assistance to urgently reinstate the regional conflict early warning and response system, in line with the following revisions:  External donors should consider providing multi-year commitments to funding field monitors, peace research projects, and CSO rapid responses, once the revised system is tested and operational.  Donors may require the CEWARN Secretariat to direct any FAU that fails to provide satisfactory quarterly accounts for expenditure to cease releasing funds for any purpose. The Secretariat should and concurrently inform the CEWERU that it may not approve any more 113 response expenditures through that FAU until the problem is corrected (this prevents failings in one member state from preventing the release of funds in all member states.  Where necessary, donors may require the CEWARN Secretariat to direct the FAU of a member state to cease releasing any funds of CEWARN Secretariat origin in response to the member state’s (financial or in-kind) contribution to the regional conflict early warning and response system falling below funds provided by the CEWARN Secretariat. Other key recommendations:  Rename peace dividend projects to peace research projects to highlight the need for regionally-relevant pilots  Encouragement/facilitation of responses at the lowest appropriate level  Encouragement/facilitation of domestication of systems, even if data flows from parts of the system are withheld at the present time 114 Annex 13: Responses to IGAD (General) and CEWARN Secretariat Comments on Draft Evaluation Report Note: Team responses have been inserted into original document in blue type. IGAD Comments on the Report of USAID Evaluation of IGAD Activities Introduction IGAD and USAID/EA held a meeting on 18 February 2016 at Lord Errol Restaurant, Nairobi to receive draft report of the third party evaluation of IGAD activities supported by USAID. During the meeting it was agreed that IGAD reviews the draft report and provides its comments to the Evaluation Team within several days after the meeting. It was also agreed that the Evaluation Team would consult internally and inform IGAD of the possibility of holding another round of consultation with some IGAD program staff to clarify some findings. This report contains IGAD comments on the draft Evaluation Report and is presented in two sections: i) General IGAD Comments; ii) Specific Comments from CEWARN. Other comments are captured in track changes and attach to this document. Also attached are additional relevant documents intended to clarify some issues raised in the report for Team’s further review. We have also attached comments on the IGAD Risk Assessment Report. I: General IGAD Comments Our understanding of the report The report has been viewed from major perspectives, which include relevance, effectiveness, sustainability, timeliness, and adequacy in terms of helping IGAD to re-focus and reflect internally. On the positive and constructive side Generally, it is a very important evaluation that will help IGAD not only to review its engagement in the three areas of the USAID support but also to examine other program areas and partnerships. It is very timely as we just launched the new five-year strategy and the findings of the evaluation will enable IGAD review its work planning and project implementation. Sections deserving reviews The purpose and the recommendations The objectives of the evaluation revolve around the issues of effectiveness (contribution to the overall IGAD objectives), sustainability (linkage to Member States regular funding), effectiveness of the supported programs (CEWARN, IDDRSI and ICPAC), adequacy of the structure of the donor coordination, and finally the USAID-partnership. We believe that the recommendations could be adequately aligned to the objectives of the evaluation. One would expect the recommendations to be articulated based on the gaps and limitations or challenges identified in the evaluation. For example, the question of whether the current donors support structure is adequate or not does not seem to be well reflected in the findings and conclusions thereby limiting the recommendations on the same. Since recommendations are informed by the findings, it is important to draw a line between the conclusions, recommendations and examine whether they speak to the initial purpose of the evaluation. Indeed, some points in the recommendation do not read like recommendations. Therefore, we suggest reviewing the recommendations in line with the major objectives and findings of the evaluation. 115 Evaluation Response: Contractually, the evaluation team must respond to the evaluative questions posed. While the team concedes that sufficient information was not always available to do so comprehensively, the report is structured directly around these questions. Conclusive statements and inconsistencies While it is important to make conclusive statements care should be taken to ensure consistencies in the report. For example, at one point, CEWARN is depicted as ‘remnant’ of previous system limited by its new Strategy Framework or that the system is stalled. At another point, it is stated that CEWARN is expanding in some countries using the new Framework. Similarly, it is challenging to understand how a system under development can at the same time be “non-functional.” In addition, we believe that some findings can be put in the right context to reduce generalizations. We suggest that the Team runs through the document to address such inconsistencies in the report. Evaluation Response: The report has now been edited to make a clear distinction between national efforts and the efforts of the CEWARN Secretariat. The evaluation team recognizes that national efforts are theoretically part of the overall CEWARN system, but this separation is made to ensure that credit is given where it is due. Qualitative Methods and Triangulation Issues The challenges dealing with qualitative issues were eminent in some sections of the report. For example, it is clear in the report that the new CEWARN Strategy Framework entails a shift in the relationships with CSOs directly involved in data collection and analysis and not CSOs in general. If the interviews (which were largely qualitative) are skewed towards these CSOs without proper triangulation, the analysis, findings, conclusions and eventual recommendations are bound to be factually inaccurate. Evaluation Response: Virtually all findings are already triangulated across multiple countries and stakeholder types. Note that when perceptions of CSOs are provided, they are clearly presented as perceptions (further editing has been done to ensure this is very clear) and it is the dominant perceptions that are highlighted. (See Annex 9.) Also note that further data, no matter what the source, is incapable of overriding or negating these respondent perceptions. While perceptions may or may not be factually accurate, they still have significant effects on operational issues, and therefore must be addressed in an evaluation such as this. Stakeholders Misconceptions: Some of the issues presented in the “stakeholder misconceptions” (Annex 10) should come to the fore of the report because, from our perspective, they relate to the biggest challenge between IGAD or its agencies, and partners and the environment. For a document this large, any reader needs to go to the very end of the document to realize the relationship between some of the challenges being referred to, and the operational context of the organization. Evaluation Response: As mentioned above, the structure of the main body of the report is contractually constrained to the specific evaluative questions posed in the USAID Task Order. There is also a page limit set on the main body of the report. The fact that these misconceptions were addressed at all, given that they do not fall directly under any specific evaluations question is an indication of how important the team thought it was to convey them. The team would also have liked to have put the annexes that contain such additional findings (e.g., annexes 9 -12) as the first annexes, but the format of the report had previously been agreed upon, thus negating this option. In short, the order of the annexes should not be taken as an indication of their perceived importance by the authors. 116 Language Issues We believe that the findings, conclusions and recommendations of the evaluations can be expressed better than currently is. Sections of the report contains very strong language that would become problematic with IGAD organs. Evaluation Response: Some language adjustments have been made where deemed appropriate. II: CEWARN Specific Comments Introduction We appreciate some of the findings of the evaluation but have serious reservations about the evaluators understanding the CEWARN Strategy Framework 2012-2019 and the new data collection and analysis system (the new system). We feel the evaluation was not able to fully grasp the new CEWARN strategic direction and, by limiting interviews to very few stakeholders within a regional mechanism that involves 7 countries and by also failing to interview CEWARN Unit technical staff involved in developing the system and not reviewing recent documents related to the development of the new system, did not endeavor to understand it. While we recognize this as a “third party evaluation,” we are concerned with the extent to which the Evaluation Team triangulated the data it received to arrive at most of its “objective conclusions” about the new system. Evaluation Response: The first sentence of the above paragraph highlights a major concern of the evaluators. This concern has also increased on reading the CEWARD Update document (see additional comments in Annex 14). This concern is that data collection and analysis tools (predominantly the REPORTER system) is being conflated with the overall (member state and regional) conflict early warning and response system. Finding suggest that the REPORTER and other systems managed by the CEWARN Secretariat do not, and possibly cannot, have significant relevance to truly rapid conflict warning and response on the ground in member states, and that member states have had to devise their own formal or informal operational systems to compensate for this. It must be recognized that a conflict early warning and response system is more than a set of data gathering and analytic tools. The current situation appears to be a case of the tail wagging the dog, with the design of the system being driven by convenience or theoretical issues associated with the design of the data management tools, whereas the tools should merely support the system rather than influence its design. This is a very dangerous situation, as it could lead to adverse perceptions among key stakeholders in member states. The evaluators recognize that the CEWARN Secretariat has the intention of improving the system’s predictive capacity, but this cannot be done at the expense of dismissal of the system’s equally important “firefighting” abilities. It is also noted that virtually all comments in relation to the issue of “objective conclusions” that were provided within the text of the draft evaluation report were effectively requiring that the report dismiss the perceptions of other key stakeholders. This is not possible, nor appropriate. These perceptions may or may not be accurate, but the fact that they exist is an important finding. The report has been reedited to further ensure perceptions are clearly indicated as such, but beyond this, it would only be appropriate to add perceptions of others, not remove existing ones. In addition, most of the information the team received about the proposed new system prior to the receipt of the CEWARN Secretariat’s update document, was drawn from a lengthy and detailed briefing from a member of the CEWARN Secretariat’s staff in Addis Ababa. Having now also reviewed the update document (see Annex 14), the evaluation team concludes that the information provided in this briefing was essentially correct, such that it does not significantly alter any findings. 117 (The evaluators made verbal requests to meet more staff members in the CEWARN secretariat, but these requests were unfruitful.) At its core, the Strategic direction does not fundamentally alter data collection and analysis system. The new system includes all elements of the old, categorizes them, and integrates new thematic areas consistent with the Strategy Framework and the 2014 Recommendations of the CPS.255 It also seeks to expand engagement of stakeholders (CSOs and Government Institutions) in data collection and analysis. These are aimed at enabling CEWARN respond to the issues raised by stakeholders during the strategy development process: open and bottom-up approach to EW, quality data and analysis, better utilization of monitoring tools in policy formulation, more engaged CSOs, improved response at various levels, emphasis on sustainability, and move from predominantly “fire-fighting”/tactical to strategic responses. The new system introduces new predictive capabilities, while the old one was mostly recording incidents of violence that have already taken place (of past trend of violence). Evaluation Response: See Annex 14 for detailed comments provided by the evaluation team in the update document itself. The report appears to slight the role of CEWARN technical and policy organs in the development of the Strategy Framework and the new system and indeed seems to doubt that the processes leading to the above instruments were consultative. We strongly urge the Evaluation Team to have audience with CEWARN Unit technical staff including being taken through the new system to correct some factual errors in the report. At the very least, we request that they schedule Skype or telephone calls. Finally, we also have serious concerns on some of the language used in the report which we fear may provoke difficult working relationship within the mechanism. Evaluation Response: It is not the intention of the evaluation team to slight any organization or individual. In fact, care has been taken to avoid doing so as it is considered an unconstructive approach. It is true that some stakeholder perceptions are less than complimentary of the CEWARN Secretariat’s approaches, but these are presented as perceptions and they remain important in their own right. The authors actively try to give the CEWARN Secretariat the benefit of the doubt where appropriate. The report even prefaces its findings regarding the new system with, “Finally, while it is recognized that the design of the proposed to system had good intentions…” Also, as most of the strong language referred to here is used in the context of stakeholder perceptions, this is a problem that must be faced regardless of this evaluation. In this document, we seek to highlight some areas that we are convinced need further refinement (we will provide comments on track changes as well). On what is changing in the old data collection and analysis system: What is changing, structurally, in the system as documented and has been approved by TCEW and CPS are: 1. Institutionalizing data collection- Moving from individual FMs to FMs situated within CSOs to enhance data quality,256 engagement with CSOs, and sustainability; 2. Strengthening analysis- the new strategy expands conflict themes and requires more expertise in analysis of different conflict drivers. Thus, it proposes a move from relying on only one (1) 255 We can provide CPS Meeting Reports if needed. 256 Refer to the attached Report of CEWARN Field Monitors’ Learning Workshop held in Nakuru, Kenya (November 2013) for lessons learned with the data quality issues in the old system. 118 NRI for analysis to at least five (5) NRIs (thematic expertise) in each Member State to enhance triangulation and analytical quality demanded by decision makers; 3. Increasing data sets- it seeks to strengthen quality of data and analysis by including additional data sets (GIS, SMS, structural, social media); 4. Appreciating dynamism of conflict drivers by moving from focusing solely on one (1) conflict type (pastoral) to other types of conflicts; 5. But, data will still be accessible to CEWERUs, NRIs, and CEWARN Unit as has been the case. Involving more stakeholders within the Information Collection Network (ICN) opens the system to wider group that ensures CEWARN remains ‘an open EW system’ 6. Response will still be undertaken as currently is but only increasing engagement with other responders. Evaluation Response: Point 1: The above does not strictly agree with the content of the formal update document. The term “field monitors” is not used in that document and when individual data collectors are indicated, these are apparently voluntary and largely untrained. This is a very different approach to current field monitors and raises a number of additional concerns. For the rest of the points above, see the evaluation team’s comments in Annex 14. On products of the new system: 1. Alerts by FMs (but within CSOs) remain; 2. Incident reports by FMs (but within CSOs) remain- but made easier to enter into the system; 3. Situation reports by FMs (but within CSOs) remain- but made easier to enter into the system; 4. Situational analyses by NRIs remains- but instead of being done by one NRI, 5 or so NRIs will generate the analysis (this is also a safeguard against bias); 5. Policy briefs to be generated as and when needed. We are aware that the Evaluation Team (the Team) has not been taken through the new system and find the conclusion that “The evaluation team also independently identified a number of concerning insertions in the proposed new system that they believe will greatly complicate the conflict early warning and response system in a manner that will promote a theoretical approach, rather than local functionality” factually inaccurate (Cf. conclusion of Evaluation Questions 1B at Para 3). It is not clear the sources of this independent verification and the specific “insertions” referred to in the statement. Unless the Team is taken through the system, they will not be able to independently confirm if the system will “promote a theoretical approach.” Evaluation Response: The formal update document includes the sentence, “In sum, CEWARN has and will continue to be guided by a theory-informed and data-driven approach to analysis.” Other inclusions in the document tend to heighten the evaluation team’s concerns in regard to this issue. See the team’s detailed comments in Annex 14. To the contrary, what CEWARN is designing, is cutting-edge, relying on similar indicator models currently applied to the AUC CEWs and ECOWARN. Evaluation Response: Are not the on-the-ground approaches developed and refined by CEWERUs and CEWARN member states at least equally important? In fact, the new system introduces new predictive capabilities, while the old one was mostly recording incidents of violence that have already taken place (of past trend of violence). Indeed, the new system 119 is “true” early warning, because it allows for prediction and trending while the old one was mostly “a firefighting engine.” Evaluation Response: The last sentence above is another very concerning statement. The CEWARN Secretariat should never simply assume that its predictive ability will be so good that it no longer needs a strong firefighting component to its system. On Sources of Information: Either by design or default, the evaluation failed to review key, particularly recent, documents related to evolution of CEWARN especially as it relates to development of the new system. Evaluation Response: Such information was requested on a number of occasions, but was not received. This challenge was well documented by the evaluation team’s administrative support team. A look at documents reviewed indicates that other than CEWARN Strategy Framework 2012-2016, there was no review of post strategy endorsement documents (Cf. to Annex 4 Matrix of Questions and Sub-Questions). These include Report of CEWARN FMs Learning Workshop, TCEWR and CPS reports (the review relied on TCEW 2009 report) including the 10th TCEWR Meeting held in Hawassa, Ethiopia in 2014 that outlined the roadmap to the new system and the subsequent reports of the December 2014 TCEWR and CPS meetings257 that approved the roadmap and piloting of the new system. Annex 6 confirms omission to get the documents from CEWARN Unit. Failure to review these documents meant that the Team could not capture the processes, including participation of diverse stakeholders, which led to the development of Strategy Framework258 and the new system. We take note that two out of seven CEWERUs responded to the Team’s formal information request, and that did not include CEWARN Unit. Evaluation Response: Note that most of the information the team received about the proposed new system prior to receipt of the update document was drawn from a lengthy and detailed briefing from a member of the CEWARN Secretariat’s staff in Addis Ababa. Having now also read the update document, the evaluation team concludes that the information provided in this briefing was essentially correct, and as such, the update document does not significantly alter any findings. Indeed, other than CEWARN Director, no CEWARN Unit program (technical) staff, especially those directly involved in the development of the new system, was interviewed. We also take note of number of former FMs and NRIs interviewed compared to CEWERU Heads and other CSOs in the region. This in our view affected the Team’s ability to adequately triangulate data especially on the “findings” on the new system. Evaluation Response: See previous response. On Reasons for changing the system: Allegations that the system may have been changed abruptly or as captured in the Executive Summary that “the full basis upon which the decision to implement this change remains unclear to the evaluation team” needs further interrogation. This statement confirms our argument that the Team was not properly informed of the processes leading the development of the Strategy Framework and the new system. The reasons for changing the system are documented in the report of the 10th CPS in 2011259 and included the need to better serve decision makers/increase use of CEWARN information by expanding thematic and geographic coverage, strengthening institutional capabilities, and extending partnerships. It was also to address 257 The referenced TCEWR Meeting Reports are available if needed. 258 See as attached. 259 See footnote 1 above on CPS Meeting Reports. 120 difficulties expressed by former Field Monitors in using the old system.260 At no point did we encounter issues of “sensitivity of information” as the reason (this allegation appears in Para 2 of the Evaluation Question 1B). And it is not a “collusion of CEWARN/Member States to ‘push out’ existing experienced CSOs and replace them with ‘tame’ CSOs.” Evaluation Response: The evaluation team reported the dominant perceptions of key stake holders. The team cannot alter these. Perceptions remain important and it is common to find that perceptions provided to independent investigators vary from those given to known interest groups. On geographic and thematic expansion: This follows recommendations of Member States as expressed through local, national, and regional consultations and endorsed by the CPS in September 2012. The statement that CEWERUs are not willing to expand and that they “were pushed into other conflict types” is factually unfounded. We have engaged with CEWERUs in many consultative processes; they have not raised such issues. Document review of the strategy development process, especially report of local and national consultations,261 will negate this statement. In actual sense, it is the CEWERUs through the TCEWR that proposed to the CPS the endorsement of the expansion agenda following their national and regional consultations. It is also interesting to note that only one CEWERU Head was interviewed to confirm this statement. Indeed, that CEWARN is ripe for expansion in thematic and geographic coverage has been demonstrated by the evaluation report at Evaluation Question 1 C exemplified by the move by some CEWERUs. Evaluation Response: See preceding response. On whether CEWARN seeks to address to all conflicts (Cf. Para 5 of Evaluation Question 1B): Why not if we understand CEWARN as a Mechanism and Strategy as a Framework? Strategy Framework foresees an expansion in stakeholders responding to early warning generated by the new system. The Mechanism is able to respond to all conflicts if all relevant stakeholders are engaged, as is the intention of the Strategy Framework. We get impression, and not just in this section but also in most part of the report, that CEWARN seems to be equated with CEWARN Unit- this misses the point. Evaluation Response: The report has been edited to make this distinction very clear. Unpopularity of the Five (5) proposed NRIs: An argument is made at Evaluation Question 1B Para 6 that the decision to select five NRIs per country, where previously there was one per country, was not popular”. During consultations leading to the development of the new system we did not get a feeling that the decision to increase NRIs was unpopular. Meetings we have held so far with CEWERUs and TCEWR (which includes CSOs) have not indicated “unpopularity” of increasing NRIs to Five. Indeed in some meetings, the CEWERUs even proposed expansion of the CSOs to be used for data collection and the NRIs to include government institutions. These proposals were presented to and approved by CPS.262 To be clear, CEWARN roadmap for the development of the new system263 (adopted by the 10th TCEWR Meeting) proposes Five NRIs as the minimum given the 5 sectors of expansion. Evaluation Response: The evaluation team is reporting the dominant perceptions of key stakeholders. The team cannot alter these. Perceptions are important and it is common to find that perceptions provided to independent investigators vary from those given to known interest groups. 260 Refer to the attached Report of CEWARN Field Monitors’ Learning Workshop held in November 2013 in Nakuru, Kenya. 261 We can provide reports of the National Consultations if required. 262 See footnote 1 above on CPS Meeting Reports. 263 See footnote 3 on TCEWR Reports 121 On Alienation of CSOs (Cf. Evaluation Question 1B Para 6): A statement appears indicating that CEWARN has “… alienated many important stakeholders, particularly interested CSOs …” The question here is “Did CEWARN conduct extensive consultation as refuted in the Conclusion of Question 1B at Para 3?” We are convinced consultation was extensive, inclusive and within the extent possible by a regional organization like CEWARN. The strategy development and subsequent development of the new system has been as consultative as it can get and has included participation of diverse stakeholders including CSOs.264 Most importantly, former NRIs and FMs and representatives of CSOs in CEWERU Committees and TCEWR have been part of the strategy and the new system development and their input to increase CSOs participation has been taken on board. The role of CSOs in CEWARN, going forward, has not been overlooked (as claimed at Question 3A Para 1) but is actually being strengthened and expanded in the new system.265 In fact, towards meeting one of the Strategic Outcomes, the new system provides room for more CSOs engagements.266 Indeed if the statement to the effect that FMs were employees of CSOs, they would still work with CEWARN but within institutional, not individual, arrangements. Further, selection of NRIs has not been done and therefore it cannot be argued that CEWARN has not left out the NRIs it partnered with in the old system. Concerning termination of FMs contracts in “a disrespectful manner” it would be important to note that at least two meetings (for example FMs Learning Workshop267 held in Nakuru in November 2013, and one jointly with Kenya CEWERU for Kenyan FMs and Peace Monitors in June 2014) were specifically held with the FMs to address transition issues before notice to terminate their contracts was issued to prepare the FMs for the transition. The contracts were even extended when the FMs still complained that the notice was not enough.268 It would also be important to acknowledge that as part of enabling the former FMs to play more roles within IGAD, the Mechanism and in the new strategic direction, the FMs are undergoing a certificate course jointly offered by CEWARN and Hekima Institute of Peace Studies and International Relations (HIPSIR); they will become members of CEWARN Facilitator Corps269 to continue supporting local level peace work. It would also add value to appreciate or even take note that despite several attempts by CEWARN, “Ex-field monitors and ex-NRIs” still felt that not enough was done to manage the transition. Further, the reference to advertisement appearing at Conclusion of Question 1B at Para 3 that “… these CSOs claim that they were not aware of the change until they saw in national media CEWARN’s advertisements seeking “new CSO partners” is not supported by facts. These advertisements only appeared in 2015 after consultations on who and how data will be collected and analyzed were concluded during the TCEWR and CPS meetings in December 2014. Participation of the “aggrieved” NRIs in the processes leading to the CPS meeting in December 2014 can be confirmed. Evaluation Response: The question here is not, “Did CEWARN conduct extensive consultations?” The real question is why the dominant perceptions and opinions encountered among key stakeholders by the evaluation team were not received by the CEWARN Secretariat’s own consultation team? The evaluation team simply reports perceptions provided by stakeholders. Given 264 See footnote 7 on reports of National Consultations 265 See footnote 3 on TCEWR Reports. Review of the new architecture vis a vis the old architecture will indicate that CEWARN is in fact expanding participation of CSOs. 266 Refer also to the section of this report indicating what is actually changing in the new system. 267 See attached workshop report. The report also details lessons learned which were taken into account in the design of the new system. 268 CEWARN can share with the Team letter informing the FMs of contract termination as well as that extending their contract. 269 This is intended in the Strategic Framework 122 the range of positions and countries from which they are drawn, the evaluation maintains that these perceptions deserve consideration and that it would be highly biased not to include them. On Strategic Framework and the new system being stalled (Cf. Conclusion to CEWARN Evaluation Question 1A): On what grounds? We are not sure that not administering functional regional conflict EWER equates to implementation of the Strategy Framework being “stalled.” Evaluation Response: The evaluators are willing to agree to disagree on this point, but the fact that CEWARN is a regional early warning, early response (EWER) mechanism was carefully considered. Proposing that it is possible to “progress” a CEWARN strategy by terminating the only working regional EWER mechanism in place was judged too incongruous a position to be credible with evaluation readers. Note, if the evaluation team had seen evidence that the proposed new system was truly being piloted (now or in the future), it may have been less concerned. But a pilot that terminates an existing system and seeks to replace it in all member states with a new system can hardly be considered a valid pilot exercise and simply reinforces stake holder perceptions that the new system is a fait accompli. Alternatively, any claims that this hiatus in the EWER mechanism is a “transition” issue (both claims are made in the comments received) would only serve to highlight extremely questionable decision making in regard to letting an existing system lapse before testing the viability of a new system. While we acknowledge that there was interruption of data collection using the regional instrument due to transitional challenges, it is important to note that Member States have continued to collect early warning using national structures. For example, Kenya continued to use its Peace Monitors, Ethiopia continued to use Federal Early Warning system, while South Sudan continued to use its CEWERs. These have continued to generate early warnings that have elicited responses. Indeed, response activities have continued based on EW emanating from CEWERUs, some CEWERUs have (as also reported in the Evaluation Report) started expanding their coverage, and engineering work of the new system continued and has just been finished.270 Evaluation Response: Largely at the request of the member state agencies, the evaluation tries to make a distinction between achievements at the national level and achievements that can be attributed to the CEWARN Secretariat. Not all CEWARN Secretariat interventions were considered helpful. Situation rooms have been established in all Member States. Most CEWERUs are indeed ready to deploy the system once the training of CSOs they have identified is done; this has been affected by delayed funding. Targeted trainings e.g. in M&E have been done. Indeed, planning by CEWERUs and CEWARN Unit since 2013 has been based on implementation of this Strategy Framework. This in our view contradicts what we understand by the word “stalled.” An acknowledgment of progress being made, albeit not as would have been preferred, would present a more balanced assessment. On allegations of “threat” to CEWARN Unit by CEWERUs improvements: This contradicts the very essence of existence of CEWARN. Horizontal learning among CEWERUs and vertical learning between CEWERUs and CEWARN Unit is in fact encouraged as the report indicates through learning visits. That some of CEWERUs lead in some areas are actually a plus to the Mechanism. Argument being advanced, at the Introduction section of Annex II: Lessons Learned, that CEWARN (we assume by this is meant CEWARN Unit) response to Kenya CEWERU’s thematic and geographic expansion was to push other CEWERUs to expand is factually inaccurate. It is not 270 For this we strongly urge the evaluation team to pay CEWARN Unit in Addis Ababa a visit to be taken through the system. We believe this visit will also enable the team to address misperceptions of system i) being textbook, academic, ii) being complicated, iii) having fundamentally departed from the previous system. 123 CEWARN Unit but the TCEWR and the CPS that “pushed other CEWERUs” to expand.271 If at all CEWARN Unit was “pushing” CEWERUs it would be to implement CPS decisions, which are prepared by the TCEWR comprising of CEWERU Heads and CSOs. But again to suggest that “While the intent of this push may be laudable, it is ill-advised in that it ignores current real world constraints and sensitivities” seems to contradict another contention in the document that CEWERUs are at different stages of development; part of the reasons they are at different stages is in recognition of those “current real world constraints and sensitivities.” Evaluation Response: See preceding response. On Achievements/Progress (Table 2): We will provide additional achievements in our track changes that have not been captured. Evaluation Response: Accepted in edits to evaluation report where substantiated. However, is it deliberate that Somalia CEWERU is left out while CEWARN has supported the CEWERU? We would understand Sudan being left out in relation to USAID policy. Are the highlighted achievement not indication of a Strategy Framework not stalled! On Early Response (including RRF): Some things need to be clarified: i) that according to CEWARN Protocol national response is anchored within Member States (the multi-actor nature of CEWERU membership is not by accident) and so the argument (Annex II: Lessons Learned at Para 2) that “National and local rapid response mechanisms therefore evolved to fill this gap by making use of locally available resources” is not accurate. Practically, this should be highlighted, as indication that CEWARN is approaching its intention- response by CEWERUs does not indicate failure of CEWARN response. Indeed, the Strategy Framework envisions CEWARN playing catalytic role (Cf. Strategy Framework at D5 “Supporting & Scaling Response Initiatives”); ii) local response is not affected by housing FMs within CSOs/CBOs. If Peace Committees’ membership includes CSOs and FMs will be housed within the CSOs, what would make local action comparatively difficult? This even contradicts a statement appearing elsewhere to the effect that the former FMs were members of CSOs. Or local action is only local if the former FMs are the ones collection data? In fact the whole response process remains as indicated in the Introduction section of Annex II: iii) at any given time, an amount of $10,000 (emergency fund) is held by FAUs for CEWERUs (unless RRF is in deficit as is currently the case) to enhance “rapidity” of rapid response- this does not require months to process and request and approval is by email to and from CEWARN Director. There are a number of cases where the emergency fund was used to prevent violence. The evaluation report ignored this and only focused on “regular fund” which requires a lengthier approval process and is often requested following the use of “emergency fund” for longer term (up to one year) prevent immediate violence from happening. Evaluation Response: The report has been amended to make its position on this matter clear. It is simply that calling a CEWARN Secretariat-held fund a “Rapid Response Fund” is misleading to potential external donors, as rapid responses are necessarily locally, or nationally-driven (and often resourced). This is not to say that the CEWARN Secretariat does not have an important role in funding longer term responses or prevention efforts as leaning exercises for dissemination to similar contexts. (Though it will need to greatly improve its existing M&E capabilities, if this is to be achieved.) To make this clearer, the evaluation now explicitly recommends that the “Rapid Response Fund” be renamed the “The Response Assistance and Research Fund.” Please note that the evaluators were made aware of the Emergency Fund by numerous stake holders, but the same stake holders noted that it was a largely inadequate solution due to ongoing replenishment issues. This emergency 271 A recent case can be found in the report of 10th CPS meeting held in Masai Mara (in Kenya) December 2014. This particular meeting “pushed CEWERUs” to play a more CPMR role at the national levels. 124 fund also did nothing to address accountability (“liquidation”) issues. This was noted in the original draft of the report. On uses of language: We find some languages (expressions) to be so strong that would make it difficult for CEWARN to make reference to the report in our TCEWR and CPS meetings and yet some of the recommendations would require CEWARN to just do so. Indeed some of the statements have the potential to create difficult working relationships within the Mechanism: for example within CEWERU Committee members (e.g. a collusion of CEWARN/Member States to ‘push out’ existing experienced CSOs and replace them with ‘tame’ CSOs; that some CEWERU members are only there to check information that is being passed across the border), between TCEWR and the CPS (e.g. that CEWERUs are not willing to expand and that they “were pushed into other conflict types” yet it is the TCEWR that presented the Strategy Framework and the new data collection and analysis system to the CPS for endorsement); etc. Evaluation Response: Some adjustments to language have been made where deemed appropriate. On Ignoring Lessons Learned: CEWARN agrees with the statement that one key lesson learned272 is that CEWARN “requires much more cooperation and input of stakeholders at the local, national and regional levels” (see Evaluation Question 1B Para 7). It is precisely because of this lesson that CEWARN strategy and new system development process has taken long. The intensive and inclusive processes that both the Strategy Framework (over 5000 people were consulted) and new data collection and analysis system has gone through, as reflected in available documents, should have been noted in the report. We take note of several “speculative” arguments being advanced by the Team at the Introduction section of Annex II: Lessons Learned and urge that they find time to first engage with the new system and then reconfirm those arguments. Evaluation Response: See detailed comments provided by the evaluation team in Annex 14. In short, the update document cites many lessons that are in agreement with the ones identified by this evaluation, but the proposed changes to the system do not consistently reflect consideration of those lessons. On “Recommendations for Revised CEWARN system” (Annex 8): We find some recommendations, specifically on FMs and NRIs, to be out of sync with requirements of the Strategy Framework and indeed with the Member States approved changes, through the CPS, and development partners concerns. For example while the Strategy Framework stresses the need for quality data and analysis, better utilization of monitoring tools in policy formulation, more engaged CSOs, and emphasis on sustainability some of the recommendations would rewind all these aspiration. Indeed, it would appear that the key recommendation is that CEWARN reverts to the old system because it “pushes out experienced CSOs.” And since the system is yet to be piloted in 2016, recommendations to the effect that CEWARN should develop a “compromise new system” (e.g. Recommendation on Evaluation Question 3B) may need to be revisited at the end of the pilot period. Evaluation Response: See detailed comments in Annex 14. Also note the evaluation is simply attempting to focus on what will work and providing recommendations that promote this. If necessary, this could imply a need for the CEWARN Secretariat to take revisions to its original proposals back to member states for reconsideration. If this is not possible, then all claims made by the CEWARN Secretariat that some form of a pilot or trial of the new system will take place are false. This scenario would validate many of the more critical stakeholder perceptions encountered by 272 Pick additional lessons learned from the FMs Learning Workshop Report 2013 125 the evaluation team. The CEWARN Secretariat cannot logically have it both ways and the evaluation team must attempt to provide logical scenarios. Such recommendations will need to be taken through TCEWR to the CPS, which has already approved the piloting of the system. But still a question that needs response is a “compromise new system” between who and who? The remedial “course correction” required to finalize the data collection system is therefore simply funding predictability. Our reading is also that funding predictability has somewhat been a challenge because the technical requirements needed to finalize the system are somewhat invisible to a layperson, and mostly related to ICT technicalities. If these are sorted out, we are good to go. Note: We believe that the Team will exercise its independent evidence based findings to review some recommendations following CEWARN comments, review of additional document we attach with these comments, and the Team’s engagement with the new system. END OF COMMENTS 126 Annex 14: Evaluation Comments on Revised CEWARN System Update (Obtained after Completion of Draft Evaluation Report, Cconverted from PDF to Word may result in format differences. Evaluation team comments are inserted in blue text.) Update on the Review of CEWERN’s Data collection Systems, Infrastructure, and Analytical Processes 10th Technical Committee on Early Warning and Response Meeting 29-31 May, 2014 Hawassa, Ethiopia 127 Executive Summary Having been the pioneering conflict early warning system on the African continent and by far the longest standing, the CEWARN Mechanism is once again reinventing itself. Based on Recommendations of the Committee of Permanent Secretaries (CPS) held in September 2012 and CEWARNs Strategy for 2012-2019, the Mechanism has embarked on the process of re￾invigorating its early warning work, aiming to transform the Reporter into a system that is easy to use, capable of accommodating more thematic data, and covering a wider geographic scope. While these changes were primarily driven by the demands of the new Strategy and the CPS Recommendations, the objective is to also bring the Mechanism at par with new technologies, and indeed to enable it to cope with the ever-changing conflict terrain in the region. Once the transitioning of the Mechanism is complete, it will not only have new and simplified user platforms, its data collection architecture will be tremendously changed to allow for broader stakeholder participation as well as more robust analysis. It is hoped that the first phase of this process of re-visioning the Mechanism will take place through to December 2014, after which a review will be conducted, to evaluate the integrity of the new system. 128 Table of Contents 1. INTRODUCTION............................................................................................................................. ............................................................... 4 2. LESSONS LEARNED IN CEWA R N’ S FIRST DECADE OF EARLY WARNING AND RESPONSE............................................................ 4 3. THE 2012-2019 CEWARN STRATEGIC MANDATE........................................................................................................................... 5 4. CEWA R N ’S EXPANDED FRAMEWORK OF PRIORITY THEMES, 2012-2019 .................................................................................. 6 5. CEWA R N ’S APPROACH TO ACHIEVING ITS EXPANDED MANDATE................................................................................................... 7 6. CEWA R N ’S EARLY WARNING AND RESPONSE ARCHITECTURE........................................................................................................ 8 7. CEWARN EARLY WARNING AND RESPONSE MECHANISM FEATURES ..........................................................................................10 8. CEWARN MECHANISM EARLY WARNING PRODUCTS ......................................................................................................................10 9. RESPONSE ACTIVITIES..............................................................................................................................................................................14 11. GOING FORWARD ...................................................................................................................................................................................16 129 1. Introduction In September 2012, the IGAD Committee of Permanent Secretaries (CPS) renewed the mandate of the Conflict Early Warning and Response Mechanism (CEWARN). Instead of focusing exclusively on monitoring trends related to conflicts in areas inhabited by pastoralist, the mechanisms’ new mandate, which is enshrined it Strategy for 2012-2019, allowed it to expand thematically and geographically. All IGAD Member States identified geographic areas and themes of interest, enabling CEWARN to take on the technical aspects of translating the new Strategy into more robust early warning system. Ever since September 2012, CEWARN has been exploring ways and means of actualizing these new responsibilities. While ensuring that CEWARN meets the imperatives of the new Strategy by collecting data that enables the prevention of violence in the region remains, the thrust of CEWARNs work also demands that it maintains a data collection mechanism that is easy to use, generates high quality data and is in tandem with ever-changing technologies. This report, which draws directly on the critical imperatives identified in the new Strategy is informed by three review meetings and lessons learned over the last decade, and is the first attempt to lay down the contours of a reinvigorated CEWARN data collection mechanism. The process of re-visioning CEWARN will continue, until the end of the year. As will be elaborated below, the Mechanism will now collect data around five conflict-related themes and a vast geographic area, while simultaneously reducing the user burden on the data collection teams. As such, the Mechanism aims to build a simpler system, which correspondingly retains data integrity. 2. Lessons Learned in CEWARN’s First Decade of Early Warning and Response CEWARN field monitoring began in mid-2003 and has been operating continuously since. The original mandate was to design, develop, deploy and operate a tool to track pastoral conflict & cooperation. The CEWARN monitoring tool (the CEWARN Reporter) supports the assessment of trends & deviations from the reported baselines on a weekly basis, with an emphasis on two initial pastoralist areas, the Karamoja and Somali “clusters.” These trend assessments are used to trigger alerts in situations that may escalate into violence. Evaluation Response: The evaluation intentionally contends that this is not actually the way it always, if ever, works in practice. The tool also tracks the overall incidence of pastoral conflict (failures when it escalates into violence) and cooperation (a measure of success), and to a limited extent the tool also monitors response efforts used to prevent or mitigate violent conflict. Over the past twelve years of operation the CEWARN Reporter has had two major upgrades, a schedule that is consistent with the life cycle of most software. In November 2013, CEWARN convened a Field Monitors’ Learning Workshop, which reviewed and captured the Field Monitors’ experiences with the system. CEWARN recently engaged area, analytic and technical experts to review the CEWARN field monitoring and analysis effort. Over two meetings in 130 April and May the review groups found that the CEWARN Reporter tool provides a theory informed, data driven framework, but with a system that is difficult to use and increasingly outdated. Specifically, the tool requires Internet connectivity; it is optimized only for a single browser type; and it is not accessible to mobile devices. Based on this recent review, CEWARN has begun to adapt the tool to serve CEWARN’s evolving mandate as well as upgrade it to better use new technologies that have become available. In addition, the review revealed that some field monitors resist the tool’s detailed, structured reporting burden. Analysts too were challenged by the tool’s complexity to the point that its credibility, especially with respect to its analysis features, have eroded. Evaluation Response: Consistent with evaluation findings. Exacerbating this perspective, the learning curve for the tool’s graphical analysis is very steep, compounded by its expansive set of explanatory indicators and its numerous parameters. The recent reviews of the CEWARN Reporter since its deployment twelve years ago yielded the following lessons learned: • Incremental improvements can facilitate modifications and training especially if they anticipate subsequent upgrades • Modular designs with can help to mitigate but not eliminate technological obsolescence • Multi-channel access and coordinated, multi-tier collection-analysis are required to engage civil society and facilitate response • Two-way communications and reciprocal incentives are required to sustain civil society participation in the CEWARN Mechanism Evaluation Response: Consistent with evaluation findings. • Simplicity contributes to credibility, evidence-based recommendations and ultimately to effective response Evaluation Response: Consistent with evaluation findings. • In sum, treat the tool as a long term investment, in need of ongoing maintenance and upgrades, and not a one-time expense • Open-source grassroots-based data collection increases community trust in the information and enhances their engagement in conflict early warning and response. Evaluation Response: The evaluation team does not disagree, and also notes that data flows from grass-root levels directly to the CEWARN Secretariat are highly sensitive issues to member states. • An effective conflict early warning and response system must have a clear in-built reporting and feedback process to aide learning, adjustment, and enhance confidence in the system and reports it generates. This document incorporates these lessons learned together with the evolving mandate of the CEWARN monitoring, analysis and response mandate. As CEWARN moves forward, however, continuity as well as the change is observed. 131 Evaluation Response: The biggest issue here is that this document conflates the CEWARN system with the REPORTER system (see Section 2 of that document). The REPORTER system has been shown to be of very limited use in regard to identifying a need for or initiating truly rapid responses. Conflating the needs of the CEWARN system and the IT system it uses as a support tool would seem to be a case of the 'tail wagging the dog'. Even the most distinctive feature of the current CEWARN field monitoring effort – its narrow focus on pastoral conflict and cooperation – was by design and was intended to be a first step toward a broader effort to follow. The expansion of this focus beyond pastoralism is discussed in the next section. 3. The 2012-2019 CEWARN Strategic Mandate Beginning with CEWARN’s 2012-2019 strategy the focus of the its field monitoring effort was expanded to include some sixty high priority conflict “typologies” or themes across the IGAD Member States. Appendix I lists the high priority conflict “typologies” from the strategy document. These high priority themes were characterized as interrelated but unique priorities as identified in each of the IGAD member States across the region, and as evident in religious, ethnic, ideological and class conflict. Among these sixty themes, twelve are usefully considered cross-cutting and/or evidenced in multiple Member States and thus may be viewed as high priority themes shared across the IGAD region. These are listed in Appendix I as “Common Themes.” In addition to these high priority themes, the strategy document emphasized integrated response activities at the local, national and international levels. The same review groups (mentioned above) that reviewed the CEWARN Reporter also examined the unique and common themes as well as mandated response activities. The groups elaborated upon sixty high priority themes to address the range of issues discussed in the strategy document. This elaboration process yielded fifteen cross-cutting themes organized across five sectors, slightly higher than the twelve common themes mentioned above. The group also outlined response activities response activities, to produce a working list of eighteen priority themes organized in five sectors with three themes each plus three levels of response or peace process activities as outlined below. Like the pastoral conflict and cooperation focus of the original CEWARN mandate, this expanded framework of priority themes serves to frame the scope of effort going forward. 4. CEWARN’s Expanded Framework of Priority Themes, 2012-2019 Sector / Activity Priority Theme Categories (18) – three in each sector & response activity Economic Commerce & Trade Disruptions & Corruption Financial Conditions & Poverty Environmental Degradation & Pollution Natural Disasters & Accidents Scarce Resource Competition Governance Due Process & Dissent Elections & Campaigning Fairness & Equality & Justice Security Armed Conflict & Violence Crime & Personal Safety Terrorism & Torture Social Identity, Gender & Tradition Pastoralism & Migration Health, Education & Quality of Life Response Local Peace Initiatives National Peace Initiatives International Peace Initiatives 132 Evaluation Response: Despite this work-shopped outcome, the evaluation found a great deal of resistance to direct involvement of the CEWARN Secretariat in many of these expanded themes by the governments and government agencies required to implement the new system. Note this does not mean that they cannot be a part of a greater CEWARN system, just that the roles (particularly in relation to primary data gathering) of the CEWARN Secretariat in relation to such themes should be very carefully selected. The CEWARN strategy document also identified better utilization of the monitoring tools in decision support for policy formulation, improved response at local, national and international levels, wider sharing of the CEWARN benchmarks and standards that promote best practices, more engagement of civil society and an emphasis on sustainability across all areas of CEWARN activities, especially in preventive response. Aside from the expanded scope beyond pastoralism, the 2012-2014 strategy guidelines are remarkably consistent with CEWARN’s original mandate. For example, the CEWARN objectives presented at its June 2002 office opening include 1) enabling Member States to prevent cross-border pastoral conflicts from developing into armed violent conflicts on a greater scale, 2) enabling local communities to play in important part in preventing violent conflicts and 3) enabling the IGAD Secretariat to pursue conflict prevention initiatives and to provide technical and financial support. Evaluation Response: It is noted that these objectives do not cover truly rapid responses. Was this limitation in the CEWARN Secretariat's real capability already recognized at this stage? In other words, both then and now, the aims include a sustainable, local analytic capacity of human resources & institutions and the engagement of civil and political societies in sustainable dialogue on evolving security situations. Likewise, with the exception of its expanded membership, the CEWARN structure has not changed significantly over the past twelve years. 5. CEWARN’s Approach to Achieving Its Expanded Mandate The most important part of any early warning system is prevention. However, CEWARN does not exist simply to warn. Its raison d'être is to prevent violence, turmoil and disruption associated with the priority themes outlined above, but when prevention fails, to coordinate the mitigation and/or resolution of the conflict led by its Member State partners. Response activities, therefore, are as important as warning for CEWARN. Also, the term “response” as used here is a bit of a misnomer in the sense that many “response” activities are actually proactive peace processes aimed at peace building and designed to prevent the escalation of conflict into violence. Evaluation Response: This again appears to recognize that truly rapid response activities are not the core business of the CEWARN Secretariat. Another important CEWARN design consideration is its transparency. Civil society￾governmental collaboration is vital to all aspects of CEWARN activities from design to operations 133 to outcomes. Indeed the software tools and protocols were jointly inspired by CEWARN, Member States and civil society during numerous and ongoing consultations. Evaluation Response: This agrees with the evaluation findings, but is at odds with some CEWARN Secretariat comments on those findings. This approach facilitates ongoing refinements to the system and also supports its sustainability. As for the sources of information used by CEWARN, they include field observations by civil society and governmental officials, along with their respective networks of local human resources. Local, regional and national media are routinely monitored by CEWARN analysts as are publications and public announcements from civil society organizations and governmental agencies. These reports on news events-oriented information are supplemented with information on the structural context, including the economic, environmental, political, security and social sectors. In other words, the reported events are contextualized within their unique frames of reference as prioritized in the 2012-2019 CEWARN strategy. Since the quality of these open sources of information, data and their subsequent analyses is critical to their value for generating early warnings, it is desirable to establish a provision for independent assessment and peer review. A policy proposal for allowing such access is included in Appendix II for review by the TCEW. This limited access policy proposal, if approved, would support the transparent validation of divergent observations and triangulation of contending analyses, views and assessments. However, certain raw data, particularly as it relates to the identity of victims of intimidation, discrimination, harassment or violence, is protected within the CEWARN system on a need to know basis. To be sure, some of the information collected by CEWARN may be highly sensitive, threatening and/or provocative, to some individuals, groups, organizations and governments. Evaluation Response: Consistent with evaluation findings. To protect the policy making process of formulating response options, and especially, to protect the victims involved in reported incidents, CEWARN system data is closed in that it is not open to the public at large. Instead policy briefs, alerts and early warning and response reports are provided by both the civil society partners as well as CEWARN itself. This open source, closed system approach is not just consistent with the transparent operation of the CEWARN system, but it is also critical to assuring that the ongoing data collection and analysis remain rigorous, systematic and credible. Evaluation Response: This simply 'glosses over' rather than addresses the security concerns of individuals, government agencies and governments. Not "not open to the public at large" and "a closed system" are meaningless statements in relation to real data security issues. The partners in the CEWARN effort include individuals, groups, organizations and agencies at the local, national and international levels. These partners include both civil society and government officials. To date, CEWARN has relied on a local information network of “Field Monitors” (FMs). These contracted individuals observed the conditions at the local level and submitted incident and situation reports to a single National Research Institute (NRI) in each IGAD Member State. These NRIs were led by a “Country Coordinator” who worked closely with their respective national government partners called CEWERUs. 134 Evaluation Response: This occurred in only some member states. In an effort to expand CEWARN’s monitoring, CEWARN will engage multiple Civil Society Organizations (CSOs) at both the local and national levels to observe and submit field reports, the form and content of which are discussed below. At the local level, selected individuals and officials would be asked to serve as the eyes and ears of CEWARN on the ground, reporting short unstructured field observations (via SMS) as events occur. In addition, selected CSOs operating at the local level would be contracted to provide weekly situation and incident reports via the web and eventually via Smart Phone forms. Evaluation Response: 1. Why are these no longer termed "field monitors" and does “asked to serve” imply that they would be engaged only on a voluntary basis? 2. If this means "eyes and ears" of the CEWARN Secretariat, with SMS messaging sent (by automatic copying or otherwise) directly to the secretariat, the evaluation finding is that some member state officials would consider this unacceptable (in relation to certain themes) and may move to render it illegal. 3. Some CEWERUs specifically voiced the need to address the legal challenges of SMS use in conflict early warning in the context of government telecommunications restrictions. At the national level, selected CSOs would also be responsible for submitting monthly situational analyses for their respective countries and certain policy briefs on issues of particular importance and/or timeliness. The multiple CSOs (perhaps five to ten in each country) would work closely with their respective CEWERUs just as the single NRIs have in the past. Evaluation Response: It is noted that no real justification of why engaging more NRIs is better than simply requiring a single contracted NRI to draw in expertise related to any theme it is not currently competent to address. The evaluation team believes this might be better both in terms of integrated data assessment and simpler contracting and probity arrangements. If the plan is that the CSO's engaged would work voluntarily, this was strongly negated by the opinions expressed by such organizations to the evaluation team. The shift here is to broaden the base and density of reporting and to supplement the national level and issue-focused analyses. No significant changes are envisioned with CEWARN’s international partners. Evaluation Response: This agrees with the evaluation finding that the new system will be of greater complexity. In sum, CEWARN has and will continue to be guided by a theory informed and data driven approach to analysis. Evaluation Response: This seems to confirm the evaluation's concerns that the new system is driven by theoretical, rather than practical, considerations and that the data collection imperatives driven by this theory appears to be 'the tail wagging the dog' when design considerations are reviewed. CEWARN’s structured field reporting supports the human analysts as they observe and record more systematically their observations and assessments. Such an approach can provide the kind of systematic measurement of precursors to violence, humanitarian crises and situations of 135 instability required of an early warning and response system. Without empirically driven events baselines, early warning alerts and measures of success are too often subjective and a matter of contention; they are based on impressions rather than systematic observation and analysis. Evaluation Response: This is a commendable statement, but does not match the partial reliance on subjective ordinal scales in some instruments (as provided in appendices). When field report baselines are developed in near-real-time (typically, these are done on a weekly basis) they can be analyzed to determine when any of the individual indicators crosses a pre-defined threshold or limit. Evaluation Response: Even if this were the case, which the evaluation finds unlikely, it is still too slow to inform effective rapid responses. When this happens, the relevant analyst is presented with a menu of pre-defined response options based on prior contingency planning. CEWARN’s integrated approach to early warning therefore provides baselines for its economic, environmental, political, security and social sector indicators. 136 6. CEWARN’s Early Warning and Response Architecture The figure below presents the architecture and information flows of the CEWARN early warning and response mechanism. Evaluation Response: The evaluation team does not understand the apparent avoidance of the term "field monitor', as the comments from the CEWARN Secretariat claim field monitors remain a key part of the new system. Are the 'individuals' referred to here only voluntarily engaged? On the next page a tabular form of the main features of the CEWARN Early Warning and Response Mechanism is presented along with notes on the level of analysis, the participant￾partners, their roles, the information products, scope and training requirements. The new and modified tools are highlighted in red. The bulk of the modifications were focused on simplifying the forms to ease the reporting burden following the recommendations of the review groups that met in April and May. The features of these tools are described in the next section. 137 7. CEWARN Early Warning and Response Mechanism Features Level Participants Number Role Information Products Scope Training International IGAD-CEWARN one Coordination of Analysis & Response and Secondary Quality Assurance Early Warning and Response Reports Regional regular professional development and training for trainers International Partners several Specialized Analyses Policy Briefs N/A National Member State CEWERUs seven Analysis & Response Early Warning and Response Reports Various annual training plus consultations as needed Non-Governmental Research Institutes about five per country Country Analysis and Primary Quality Assurance Country-Month Situational Analyses National orientation plus annual training and training for trainers National Partners several Specialized Analyses Policy Briefs Various N/A Local Selected CSOs up to tens per country Information Collection & Monitoring Weekly Situation Reports Various Areas of Reporting (AORs) annual joint training by CEWARN & national CSOs Selected CSOs up to tens per country Information Collection & Monitoring Episodic Incident Reports Selected Individuals & Officials up to hundreds per country Information Collection & Monitoring Episodic Field Observations annual orientation plus web accessible quick guides Evaluation Response: This table clearly implies that the individuals now involved in the system will be not be fully trained, as was at least the intent with 'field monitors' under the pre-existing system. Considering the risks to these individuals and the hard-won trust built in the communities around these existing individuals, this would seem a very risk laden approach, both for the individual and the system. 8. CEWARN Mechanism Early Warning Products At the local level, three information collection and monitoring tools are to be supported to generate early warning products. Two additional products are generated at the national level, one of which may also be addressed to the IGAD regional level as follows: 138 Product Primary Level Field Observations local Incident Reports local Situation Reports local Situational Analyses national Policy Briefs national and regional Although primarily used to monitor the local level, field observations, incident reports and situation reports may be submitted by partners at any level as needed. Field Observations or Fobs are the most simple of the three information products, as they contain only about 150 characters of text and are submitted via SMS. Fobs are to be used for any type of communication that is urgent, including warnings, reports of imminent violence and the like. They might also include a short note submitted after an incident occurred to report casualties or request assistance. CEWARN envisions the selection of up to a thousand individuals and local officials per Member State to assist the information collection and monitoring effort through the episodic submission of Fobs. Evaluation Response: This alone would imply that they will need to perform their duties on a voluntary basis. Again this is very risk laden in terms of consistently acquiring timely sufficient and accurate observations. Although typically unstructured in their content, basic information can be automatically (or with human assistance) extracted from them, including the “who did what to whom, when, where and how.” Evaluation Response: These are high quality, contextually rich and verifiable, raw data. These parameters are a subset of similar information contained in incident reports. This congruence allows one to graphically and geographically present the two data types together on time series graphs and maps. Given the ubiquity of cell phones and especially the use of SMS texting in the region, training of these participants is expected to be minimal. Evaluation Response: This focus on minimal training again confirms that ‘individuals’ under this system are very different to Field Monitors under the pre-existing system. CEWARN is to prepare a web-accessible quick guide and/or tutorial for the participants asked to submit fobs. CEWARN as well as its national partners will also be available to answer questions should they arrive via email and phone. For the most part, however, little more than an orientation and the quick guide and/or tutorial are planned for training. Evaluation Response: See preceding comment. 139 The Fobs are a new feature of the CEWARN Reporter, introduced to improve the information collection and monitoring at the local level. The Fobs will be managed by each Member State directly with a pass through copy forwarded to and retained by the CEWARN Unit. All Fobs users will be given a national toll free number to submit their Fobs to their respective CEWERU. Yet to be determined is the extent to which the Fobs participants will receive alerts or other early warning information from CEWARN or the associated CEWERU via a response SMS sent to their cell phones. Evaluation Response: "Pass through copy” implies direct acquisition of this data by the CEWARN Secretariat. Again, this could be a problem in relation to national sensitivities related to certain proposed themes, and possibly member state telecommunications legislation. Incident Reports or IncReps generally contain an expanded version of the basic parameters, the “who did what to/with whom, when, where and how.” Evaluation Response: These are high quality (contextually rich & verifiable) raw data. CEWARN’s current Incident Report (IncRep) form has approximately fifty parameters and is perceived as burdensome and complex by many users. A simplified version of this IncRep form has been drafted with the help of the review groups mentioned above. The simplified version has about half the number of parameters and is expected to be easier to use. A draft listing of the simplified IncRep parameters is included in Appendix III. Note that all of the forms included in the Appendix are works in progress and are to be refined as the application design transitions into engineering by next month. Evaluation Response: This document was created in mid-2014. Are there later versions of these instruments drafted? This would seem a highly necessary step if the system is to be progressed. If not, then the evaluation team’s concerns regarding implementation of the new system appear further justified. What qualifies as an incident to be reported remains to be formalized. CEWARN’s current IncRep form identifies incidents relevant only to pastoral conflict. A broader set of reportable event types is under development to address the expanded mandate of priority themes. Preliminary guidelines for this process are outlined below. 140 IncReps are to be submitted as soon after an incident occurs that is within the pre-defined set of eligible incidents. Eligible incidents are framed by the priority themes outlined above. Additional criteria are used to determine whether an eligible incident is reportable: 1. An incident involves or is related to one of the fifteen priority themes AND 2. An incident is associated with a negative polarity loss or otherwise “bad” outcome AND 3. An incident involves one of the priority event types (see below) OR 4. In some other way the incident is significant (an explanation must be included) Evaluation Response: If CEWARN has an interest in addressing 'unpredicted' sources of conflict, it would likely be most effective to also include this last option as a 16th "Other" category of priority themes. Not including it on an equal footing with the stated priority themes may bias against this category of incident being reported. An example of the fourth, discretionary criterion might be a tribal chief calling for a rally to support a policy initiative being actively considered by a legislative body. In this case, the justification would be that although the proximate outcome may be positive (a support rally) any resistance to the initiative is likely to result in a negative outcome. Situation Reports or SitReps, in contrast to the episodic IncReps, are submitted on a regular basis to describe the conditions within a given area of responsibility or reporting (AOR). Evaluation Response: It is difficult to see this happening, if they remain focused on subjective rating scales. SitReps generally focus on an AOR that may span an entire country or be limited to a region within a country. The time periods for SitReps range from a week to a month or more. CEWARN is proposing a continuation of its current weekly SitReps. Evaluation Response: It would be interesting to see the hard data on how regularly SitReps have been done to date. The actual indicators within a SitRep are specific to the mandate, and in the case of CEWARN, they evolved from the review meetings held in April and May. The indicators also build upon CEWARN’s decade of experience with its current SitRep form, but with a considerable simplification to address the issues of user burden and data quality. In addition to the structured responses in the form, CSOs are encouraged to include annotations, notes or unstructured narratives to amplify or clarify their structured SitRep observations. Given that situation reporting may cover a wide range of themes, it does not typically enjoy a common base set of parameters for its content. Specifically, there is no equivalent to the incident report’s “who did what to/with whom…” base parameters. Instead, each application of a field situation report is guided and informed by a protocol or set of assumptions, terms, procedures and rules specific to its use. Departing from the current narrowly focused SitRep 141 indicators the proposed CEWARN indicators have been drafted by the review groups. These draft simplified SitRep indicators are presented in Appendix IV. SitReps are useful in monitoring pre-cursors to escalation of conflict. By linking the information collected in SitReps with the monitoring of “failures with IncReps, one is able to identify the precursors in the SitReps that led to the “failures” or escalation of conflict as recorded in the IncReps. This integrated use of SitReps and IncReps, together with an appropriate time lag, is the linkage that facilitates early warnings. Evaluation Response: The “who did what to/with whom…” data are the real raw data and as such, it is important to structure analyses making use of it. Leaving it up to the respondent, by defaulting to subjective rating scales in SitRep summaries, is a means of avoiding consideration of what form such analyses should take. SitReps can be used with a wide range of dynamic indicators that presage conflict escalation. Draft indicators have been developed but they also need to be refined. Evaluation Response: What are these dynamic indicators and how is information relating to them gathered? Operationalization and scaling of the indicators during a field testing phase is required to refine the customization process. Indicators may include both positive, mitigating factors as well as negative or exacerbating factors. In this way, one can be informed of peace-conducive and conflict resolution initiatives. This approach also facilitates the drafting of specific response strategies that build upon local peace initiatives. It is the inclusion of SitReps that makes this integrated, structured report methodology a proven approach for early warning and response. In sum, the integrated use of situation and incident monitoring produces reliable real-time baselines of field activity and behavior across a set of high priority themes in the region. This approach enables users to compare baselines over time and to anticipate behavioral change that may lead to conflict. Such changes or inflections in behavioral baselines may signify a deviation from the “norm” which, if undesirable, may indicate a need for early response to prevent conflict or other failure in human security. Situational Analyses are intended for use at the national level. They are to be conducted by CSOs contracted by the CEWARN Unit similar to the work being done now by the NRI within each country. However, going forward CEWARN plans to engage multiple CSOs within each Members State to complete country-month situational analyses. The redundancy here is by design – to help triangulate and enrich the various analyses. A draft outline of the proposed Situational Analyses form is included in Appendix V. The objective for the national non-governmental research institutes (NGRIs) here is to draw upon the field observations, incident reports and situation reports submitted over the preceding four weeks to assess situation in the country over the past month; thus these situational analyses generate country-month reports. Evaluation Response: The situational analyses appear to focus on generating responses to subjective rating scales. To be meaningful on a regional basis, each point on every scale would need to be unambiguously defined. This is difficult to do in practice and if it can be done, begs the question, 142 why not just use the criteria used in the definitions as comparative measures, which would negate the need for any scales, other than as a form of 'dashboard' or 'traffic light' representation, which is their real value. These country-month situational analyses represent the primary means of tracking the baseline activities relevant to conflict and cooperation in each country. Especially when viewed in a time series graph over a full year of seasons, the deviations from these baselines offer an early warning of potential escalation. These same time series also offer a means of tracking the progress within a country in terms of the status of the themes within the sectors as well as the response (peace initiative) activities at the local, national and international levels. The indicators for escalation potential, urgency and severity likewise offer timely, regular and comparable assessments that reflect the conditions on the ground across the priority themes being monitored. The forms also encourage the entry of narratives and notes to supplement and contextualize the ratings. In sum, situational analyses facilitate evidence￾based, quantitative and qualitative assessments of evolving situations that may escalate into violence, turmoil or disruption across the economic, environmental, governance, security and social sectors as well as any response activities. Policy Briefs are intended for use at the national level, but they may also be used to report on cross-cutting issues across the IGAD region or beyond. These briefs may be focused on a prominent activity such as an election, or on emerging problems such as climate change or terrorism or gender inequality. National policy briefs may be solicited by CEWARN from CSOs that specialize in the topic of interest. At the regional level, CEWARN may produce certain policy briefs internally as well as outsource them to CSOs who specialize in the chose topic at the international level. 9. Response Activities The primary responsibility for response activities is anchored in the CEWERUs at the national level. The actual initiation of response may occur at the local or international (especially regional) levels, but it is the national (CEWERU) units that monitor and coordinate all response activities within their respective countries, whereas CEWARN unit plays a pivotal role through coordinating the peace processes at the regional (international) level. Evaluation Response: In agreement with evaluation findings. To do so, CEWERU need to engage local and national officials as well as civil society organizations within their respective countries in an ongoing dialogue on peace process. The CSO partners contracted by CEWARN Unit within each country serve as liaisons in this dialogue between the local and national levels as well as with the CEWARN Unit. Although each Member State is likely to be somewhat different, the local level peace process activities are expected to be initiated by civil society organizations and/or community based organization and/or the local leadership in the area as well as the traditional dispute resolution 143 forums where it exists, CEWARN serving in a support role, particularly when a regional or international response is needed to prevent an imminent conflict or de-escalate violence through its regional coordination bodies; Technical Committee for Early Warning (TCEW) and the Committee of Permanent Secretaries (CPS). Evaluation Response: In agreement with findings and recommendations of evaluation. The biggest challenge of the time for any early warning and early response system is to demonstrate the capacity to provide timely, effective and efficient responses to prevent violent conflict. Evaluation Response: In agreement with evaluation findings. CEWARN as a regional body has successfully developed a primary source of early warning capacity and it became a strategic imperative to link this capacity with an appropriate response components. Evaluation Response: Intentionally disputed by evaluation, as REPORTER system and other activities of the regional body of CEWARN are not in a position to inform or assist in truly ‘rapid’ responses. To this end, in close collaboration with its supporters, CEWARN has launched a Rapid Response Fund (RRF) in January 2009 with an aim to support peace building initiatives on Conflict Prevention, Management and Resolution (CPMR) at all levels. Some of the activities supported by RRF are; emergency support to local peace processes, creating joint access to local resources and capacity building, peace dialogue, technical and applied research on peace processes. (See Appendix VII) Evaluation Response: This list appears to support the evaluation’s assertion the CEWARN regional body is not in a position to inform or carry out truly rapid responses. The newly designed situational analysis report is meant to track the status and role of response activities at all three levels (local, national and international) in real time and the extent to which these responses are successful in achieving their ultimate objectives. Furthermore, the situational analyses are intended to support diagnoses by the analysts in the identifying peace process activities that address the specific issues and problems on the ground. By assessing the response activities at all three levels in each monthly situational analysis, the analyst, may be able to identify any potential connections and synergies across the levels of response. In short, the regular monitoring and analysis of response activities can contribute to integrated response strategies with synergies among the levels of peace initiatives and the parties engaged in them. More importantly, the data collected through CEWARN reporter will be shared in a real-time with respective CEWERUs at all levels, which creates a critical linkage between the situational analyses report and the formulation Evaluation Response: How and with what data security measures incorporated? Through time, the data on peace process will eventually serve as a baseline for future peace process scenario building or to document the best practice in peace process in the region. On top of that, this process will help all stakeholders and decision makers at all levels to take an appropriate actions based on timely and accurate data. 144 Evaluation Response: The evaluation intentionally disputes that the REPORTER system has been, or can be, capable of informing truly rapid responses. These were reported as being predominantly based on informal and local reporting structures managed directly by CEWERUs. 10. Quality Assurance, Training, Monitoring and Evaluation Quality assurance arguably is the single most important activity for all participants in the CEWARN field monitoring and analysis effort. The aphorism, “garbage in, garbage out,” cannot be overstated, especially when the information monitoring and collection is distributed across a region the size of IGAD. Evaluation Response: Agreed. See cautions on collecting subjective ratings as key “indicators.” In general, participants who are collecting or monitoring information are responsible for understanding the requirements of their respective tasks. Although these tasks vary from person to person, the general procedures for collecting data may be characterized as follows: Understand the purpose for which the information is being collected Identify all relevant information sources Qualify or annotate the information as needed Document the above steps in the appropriate text boxes in the various tools The Incident and Situation Reports (IncRep and SitRep) from ECOWARN are used as evidence for the situational analyses so care should be taken in completing them. Other sources of information include the media, local officials, civil society (including informal) networks and the international community. Even though the situation reports are to be done weekly, it is important to note that information collection and monitoring is a continuous process and it is to be done on regular basis to meet the report submission deadlines. For any data to be meaningful for early warning its quality needs to be verified when collected. Evaluation Response: Again, we need to be careful with use of subjective rating scales, as these are, by definition, unverifiable independently of the specific respondent(s). This is to avoid any inconsistencies and other anomalies with the data thereby avoiding erroneous reports. Data quality is a shared responsibility throughout the entire collection, monitoring and analysis processes. Consistency and reliability of the information, timelines of its submission and its accuracy are all required for this early warning and response effort to succeed. Another requirement for success is regular training. This training is critical to supplement the quality assurance as well as to promote professional development of all participants. Evaluation Response: This is at odds with above statements to the effect that individual data gatherers will be given minimal training. At least annual training is in order for all participants, with more extensive training needed at the national level where the situational analyses are conducted. The CEWARN Unit, as 145 coordinator for the entire effort, is the lead trainer, but CEWARN staff members also need regular training, some of it oriented to them as trainers, or in other words, training of trainers. Clear documentation is also required for all tools and procedures. Ideally this documentation should be as visual and as simple as possible while succinctly conveying the necessary information. Examples are useful too, as are FAQs and quick guides. As for Monitoring and Evaluation or M&E as it is called, the CEWARN Unit plans to implement a regular procedure beginning at the end of the field test period with a review of the initial deployment. Evaluation Response: Consider conducting a trial on the M&E data collection at the same time as field testing other data collecting instruments? At least annual repeats of this review are also planned, some of them internal in the form of a peer review and some involving external, expert participants with professional experience that may be brought to bear on the evaluation. Ultimately, the performance of the systems will be assessed based on: i) the extent to which it is capturing potential violent conflicts in areas of coverage (i.e. are there conflicts happening in the area of reporting that the Reporter is failing to capture?). Evaluation Response: Success here must not be seen as solely about the REPORTER system. It is possible to read this entire document as pertaining to an upgrade of the REPORTER system, rather than the CEWARN System. The two must not be conflated, as the CEWARN System encompasses all operational and aspirational elements of conflict early warning and response, and the REPORTER system is simply an IT tool employed to assist in some aspects of data management. For example, what does it mean if violence erupts in CEWARN area of coverage but the Reporter failed to give a warning about it? This will be a key marker of whether or not the system is achieving its intention; ii) conflicts alerts/warnings responded to (leading either to prevention, mitigation, or resolution). In our metrics, we intend to track # of violent conflicts occurring as a measure of CEWARN's early warning and preventive/ response initiatives. 11. Going Forward From the foregoing, it is clear that the CEWARN Mechanism is transitioning into an unrivalled data collection, analysis and response system. Having been a continental pioneer in the area of conflict early warning, the Mechanisms is once again blazing a trail in developing a path- breaking system. Just like any transition however, period between June and December 2014 will be creating. As noted in the work plan provided in Appendix VI below, a tremendous amount of engineering work will have to be undertaken alongside other preparatory activities, for instance refining the indicators, developing coding handbooks, and undertaking training for both the staff of the Unit. These latter activities may have to be undertaken once the system has been piloted but they need not necessarily proceed in the order provided. Evaluation Response: The last sentence contains the only (variant) use of the term 'pilot' in entire document. Yet it is not clear if this is referring to just REPORTER system. Many comments from the 146 CEWARN Secretariat on the draft evaluation report noted that they are simply ‘piloting’ a new system. This does not correspond with stakeholder feedback, nor with the fact that the pre-existing system has already been discontinued. Appendix I: High Priority Conflict “Typologies” (Themes) from the 2012-2019 CEWARN Strategy Country Djibouti Djibouti Priority Conflict "Typologies" (from the Strategy Framework) Competition over resources Conflicts caused by environmental pressures/climate change Common Themes Environment & Resources Environment & Resources Djibouti Entrenching peace through curriculum development and delivery Peace Processes Djibouti Gender-based violence Gender Djibouti Inequality Inequality Djibouti Murder/killing – inter-personal, intra/inter-group (tribes, pastoral groups) Safety & Security Ethiopia Ethiopia Border/territorial conflict Drought Borders & Boundaries Environment & Resources Ethiopia Imbalanced development Inequality Ethiopia Ethiopia Inter-ethnic conflict Land grabbing Identity Borders & Boundaries Ethiopia Migration (In country and cross border) Migration Ethiopia Piracy Safety & Security Ethiopia Proliferation of SALW Safety & Security Ethiopia Religious-based conflict Identity Ethiopia Resource-based conflict (climate-driven conflicts; land related conflicts) Environment & Resources Ethiopia Terrorism (e.g., violent extremism) Safety & Security Ethiopia Violence between pastoralist communities and commercial cattle rustling Pastoralism Kenya Boundary and border disputes Borders & Boundaries 147 Kenya Conflict sensitive development (Infrastructure, large-scale investment) Commerce & Trade Kenya Elections violence and devolution Governance Kenya Kenya Kenya IDPs, Migration and Refugees Land ownership Natural resource exploitation (oil, minerals, pasture, water…) Migration Borders & Boundaries Borders & Boundaries Kenya Organized crime Safety & Security Kenya Proliferation of SALW Safety & Security Kenya Urbanization Migration Kenya Somalia Violence between pastoralist communities Clan based conflict Pastoralism Identity Somalia Climate change induced conflicts Environment & Resources Somalia Somalia Somalia Control of trade and revenue routes Food insecurity Land ownership and boundaries Commerce & Trade Environment & Resources Borders & Boundaries Somalia Livestock rustling Safety & Security Somalia Somalia Somalia Refugees and IDPs Resource-based conflict (minerals, scarcity of pasture and water) Trade Migration Environment & Resources Commerce & Trade 148 South Sudan South Sudan South Sudan South Sudan South Sudan South Sudan Sudan Sudan Sudan Sudan Sudan Border and boundary conflicts Child abduction Constitutional (statutory versus Customary Laws) Crime Land ownership disputes Violence between pastoralist communities Borders and boundaries Conflicts due to pressure from climate change Land grabbing Land use and laws governing land Natural resources use and management Borders & Boundaries Gender Borders & Boundaries Safety & Security Borders & Boundaries Pastoralism Borders & Boundaries Environment & Resources Borders & Boundaries Borders & Boundaries Environment & Resources Sudan Proliferation of SALW Safety & Security Sudan Uganda Violence between pastoralist communities Border disputes (domestic and international) Pastoralism Borders & Boundaries Uganda Uganda Uganda Election-driven violent conflict Environmental and climatic drivers Food insecurity Governance Environment & Resources Environment & Resources Uganda Land (conflicts arising from population growth, investment) Borders & Boundaries Uganda Marginalization Inequality Uganda Migration Migration 149 Uganda Migration (IDPs, foreign migrants, population growth, and people resettled through large scale investment and development projects) Migration Uganda Natural resource competition (minerals, oil & gas, forests, pasture, water) Environment & Resources Uganda Negative ethnicity Identity Uganda Violence between pastoralist communities (continuity/integrate with resource/land issues) Pastoralism 150 Appendix II: CEWARN Data Access Proposal Evaluation Response: Very notable that this section remains blank in the document provided. 151 Appendix III: Simplified IncReps – Draft Parameters Evaluation Response: Sound contextual raw data collected here. Category Parameter Normalized Selections plus Text Boxes Who Age (levels TBD), Unknown OK, Mixed OK Who Gender (levels TBD), Unknown OK, Mixed OK Who identity (ethnicity, religion, etc.) text box Who number involved text box Who organization / affilitation / title text box Whom Age (levels TBD), Unknown OK, Mixed OK Whom Gender (levels TBD), Unknown OK, Mixed OK Whom identity (ethnicity, religion, etc.) text box Whom people affected: deaths, injuries, diplaced text box Whom organization / affilitation; text box What incident type selectable incident type, organized by the priority themes for negative polaitry incidents only When Date selectable from calendar Where Location GIS normalized Where geographic scope isolated, scattered, contained widespread Why Trigger text box Why Significance text box Outcome destruction, deaths, injuries & other costs selectable from categories Outcome immediate response text box Meta Source credibility (levels TBD), Unknown OK Meta Source of report (levels TBD), Unknown OK Meta all linked information N/A 152 Appendix IV: Simplified SitReps – Draft Indicators Evaluation Response: Only subjective ‘indicators’ appear to be collected here. Suggest using objective indicators. 153 Appendix V: Situational Analyses – Draft Indicators Evaluation Response: Only subjective ‘indicators’ appear to be collected here. Suggest using objective indicators. 154 155 Appendix VI Work Plan Evaluation Response: This work plan is all about the REPORTER system, rather than the CEWARN system. The IT 'tail' should not be permitted to wag the much broader operational 'dog'. Any effective CEWARN system must be much more than just data gathering and theoretical analyses. Note also 'field testing' does not equate to a 'pilot' or 'trial'. Field testing generally relates to data collection instruments alone. Testing these does not imply that the proposed new system is open to being considered inappropriate and rescinded. Many comments from the IGAD Secretariat suggested merely an intent to ‘pilot/trial’ the new system. This is not in accord with stakeholder perceptions, nor the fact that the pre￾existing system has already been terminated, nor the fact that the ‘pilot’ is being carried out in a blanket fashion across all member states. The claim cannot be considered credible by the evaluation. 156 157 Annex 15: Additional Comments on IGAD Evaluation Report from the CEWERUs Note from the Evaluation Team: This annex was added to give visibility to comments that arrived after the report had been finalized. It should be noted that the evaluation was completed in October 2015 so any activities that occurred within IGAD after the completion of the evaluation, are not reflected within this report. The evaluators reviewed these comments carefully and did not consider them to have any impact on major findings or recommendations. However, it is important to correct a misconception that appears to run through these comments from CEWERUs. To clarify: the Evaluation report does not claim that conflict early warning and response activities have stalled at national levels, nor that all member state CEWERUs have abandoned use of field monitors. The report is careful to refer to only the 'regional' system as stalled. In fact, the report also points out that CEWERUs have made significant achievements despite problems at the regional level -not least the problems associated with the attempted introduction of a new, untested system at the expense of the already operational one currently used by CEWERUs. Following are the comments from the CEWERUs: Introduction CEWARN Unit already presented its response to the DRAFT Evaluation Report through IGAD on 25 February 2016 hopes either the comments have been incorporated in to a final report to be shared or there is still plan for the Evaluation Team to visit CEWARN Unit to engage with the new data collection and analysis system. However, invoking their right of reply, CEWERUs asked CEWARN Unit to share with them the Draft Evaluation Report for their response. Consequently, we have consolidated the response as presented. A) Djibouti CEWERU  The Evaluation Mission arrived in Djibouti in the month of August 2015 without prior notice to the CEWERU of Djibouti. All CEWERU officials were on leave outside of the country since the month of August is known to be particularly hot in Djibouti. The officials of the CEWERU of Djibouti have not been found as it is written in the report. They were officially on leave. I think for the success of a mission of such importance, the CEWERU of Djibouti should have been given a notice ahead of time to enable it to make the necessary arrangements to accommodate the mission or even postpone or cancel their annual leave. Note: 1. It is written in the report that the CEWERU of Djibouti has not conducted activities since 2013. This does not reflect the reality. Without being exhaustive, here are the following activities undertaken by the CEWERU of Djibouti in 2015: a) A training workshop on monitoring and evaluation to the members of the Steering Committee from the 3rd to the 5th of August 2015. b) A study on mapping conflicts areas or disputes in Djibouti between June 2015 and September 2015 c) Selection of associations as part of the framework for a partnership in the implementation of information gathering network with the civil society between June and October 2015. d) Establishment of a Situation Room (non-operational) and the receipt of equipment delivered by CEWARN. 158 B) Ethiopia CEWERU In general terms the evaluation report might be a baseline for our future assigned duties and responsibilities. Despite this, we do have some comments on some issues: 1. The New approach of CEWARN system----Data Collection With regard to the new proposed data system, as the researcher put it, there has been some concerns among CEWERUs. But the way articulated and concluded in the document seems to lack a sort of reality. Well, all CEWERUS have concerns about losing the previous Field Monitors. But it doesn’t mean that the system significantly close the room for involving CSOs. In our understanding, the five thematic areas of conflict involve a number of CSOs at national and local level as compared to the previous system. So, the conclusion of the researcher in line with the involvement of CSOs seems to contradict with the strategic document of the system. The concern raised by CEWERUs has been the incentive of many CSOs involved in the system. 2. Ethiopian CEWERU The researcher rightly put it that the Ethiopian CEWERU had support from multiple sources as well, including UNDP, Pact, and GIZ. The sentence “this support however, does not involve maintaining field monitors” has to be corrected. Because, we have field monitors not only at CEWARN areas of reporting but also in most conflict prone areas. We had continuous training of field monitors. The resource has been from these listed donors in addition to government budget. 3. Preferential Treatment In the document, we have come across that there is a perception of unequal treatment of CEWERUs. But, in my understanding the researcher should verify it against the project submitted, the finance reimbursed and the capacity of liquidation of the allocated budget of each CEWERUS. We have concern that whether the research team had serious discussion at the CEWARN office on this issue. We recommend revising it again. 4. Government Budget The CEWARN structure is a network of government and non-government bodies. Especially, at CEWERU level, a key institution is government. We agree that some of the Member State are not committed enough to fulfill their financial contributions. Despite this, we believe that they allocate annual budget for the activities of peace at national level, which is considered as the budget of each CEWERUs. The researcher should also take in to consideration the implication of the government national budget towards the system. C) Kenya CEWERU 1. Evaluation Question 1 A: To what extent do the various IGAD efforts supported by USAID coordinate to achieve IGADs overall objectives? On client capture- The CEWERU concurs. Such perceptions are not entirely misplaced. This matter especially has left a very bitter taste mainly in the mouth of former Kenyan FMs which has resulted in reinforcing perceptions that since most of the key staff of CEWARN are from the country, the prolonged engagement of Ethiopian FMs etc. while the contracts of the rest in the region had been terminated is being attributed solely to this factor. 159 Participatory monitoring and evaluation and learning – Qualifies that the training on M&E was on CEWARN MERL Framework. Further notes that the CEWERU already has a working M&E framework developed with support of UNDP. Decision Quality – The CEWERU questioned the new approach considering that already CEWARN has been unable to sustainably support a single NRI. Extent of use of 5 or so CSOs, whose work already hinges on support for free information, is a challenge. Already, very few CSOs in Kenya have expressed interest, and even then, they are awaiting to see whether CEWARN would offer some incentives to facilitate their effective participation in the system. Conclusion – While the implementation may not be considered as having stalled, it is important to note that very negligible progress has been noted since the Strategy was launched. It is therefore recommended that at this moment, CEWARN consider undertaking a mid-term review of the Strategy in order to establish progress made since inception of the strategy in 2012. 2. Evaluation Question 1 B: What has worked well and what has not? At footnote 84 provides additional examples. For example, in 2010, while adapting the CEWARN system into Kenya context, Kenya CEWERU expanded coverage both geographically and typologically of the new system by deploying a constellation of Peace Monitors (PMs) in areas hitherto outside the CEWARN areas of reporting in order to respond to the unique needs of the country, with a lot of success. Based on this evident success, the Kenyan Government in 2012 upgraded the CEWERU Unit into a fully-fledged Directorate of Peacebuilding and Conflict Management with a view to sustaining this framework. In the same year, the Government approved the National Policy on Peacebuilding and Conflict Management as well as a full staff complement by adopting a Scheme of service for Peacebuilding and Conflict Management personnel. CEWARN Unit should have encouraged other CEWERUs to follow this example. However, the new system appears to be responding to the imperatives of shrinking donor support to the institution rather than out of outright failure of the initial system. CEWARN fixation with the system is attributed to this approach, a situation that was well noted by the IGAD Committee of Permanent Secretaries (CPS) in 2014 and the Emergency CEWARN-CEWERU Consultative meeting of May 2015 in Mombasa, Kenya which asked CEWARN to first pilot the system before gradually rolling it out in the Region. At footnote 99 provides clarification. CEWERUs are not resistant to expanding. Indeed, as noted above, Kenya expanded in 2010 in both aspects of geographical coverage and typology of conflicts even before CEWARN undertook the medium-Term review of the initial strategy, a decision which provided a catalyst for development of the new strategy (2012/19). It is the alacrity with which CEWARN appears ready to abandon what has been working instead of building on it due to donor demands that is resulting in CEWERUs taking much longer to engage fully with the system. At footnote 101 further expounds that RRF just complements other national CEWERU response initiatives. However, the manner and procedure of its administration makes it not a reliable “rapid’ response initiative. The procedures are cumbersome and inordinately slow, resulting in it not meeting the intended purpose in the long run. At Conclusion paragraph 2, It is a credit to CEWARN that the capacity of CEWERUs in the region has been growing exponentially, somehow outstripping the capacity of the CEWARN Unit. CEWARN should therefore take advantage of this success story and work for replication of best practices in the region as well as adapt its support to the unique needs of each CEWERU in the region rather than adopting a one-cloth-fit-all approach as it is currently been trying to do. This way, the cumulative outputs of the 160 CEWERUs will position the mechanism as the premier conflict early warning and response system in the continent. D) South Sudan CEWERU 1. On progress by country against Strategic Pillars (Table 2) i. South Sudan CEWERU received training in conflict analysis and conflict early warning and also participated in familiarization visit Ethiopia and though more visit within the region is yet desired. ii. Equipment brought by CEWARN such as GPS, Internet server are not yet fully in use due to limited funds from CEWARN – however, the equipment if fully functioning will boost CEWERU operation. GIS training was done in Entebbe but require further training in the ground. 2. Evaluation Questions 2 B: What have been the main challenges for CEWARN and the CEWARUs in addressing cross border conflicts? The CEWARN’s early warning system reorientation of the data collection approach is a major challenge. And as data collection by field monitors has been stopped (due to termination of their contracts). For South Sudan the FMs are still very important – The SMS system still face challenge due to insecurity implication and clearance by the National security. Additionally, uncoordinated state responses have left stabilized communities vulnerable to their cross￾border neighbors. For example, Uganda pursued a sustained disarmament and development program, while South Sudan did not, and Kenya only temporarily implemented it. (South Sudan is trying to develop comprehensive approach for disarmament –this approach will be coordinated by DDR and SSPRC, in 2010 disarmament was done especially in Jonglie State and in Juba SPLA normally do random disarmament) – There is an attempt for disarmament. 3. Recommendation i. As Mentioned in the report some countries need more support – South Sudan for example requires more support from CEWARN – both technically and financially to fully realize the strategic objectives set, however, South Sudan is on track to implement the strategic objectives though there is limited support. ii. South Sudan CEWERU confirm the issue of slow disbursement of RRF iii. CEWARN supported South Sudan CEWERU to expand its operation to Northern Bahr El Ghazel and Western Equatoria States but require further support in terms of training and familiarization into the Conflict Early Warning and Early Response system. E) Uganda CEWERU 1. On progress by country against Strategic Pillars (Table 2) 161 The table left out what Uganda has achieved in the areas of Monitoring and Evaluation. Here we had two officers trained and they were able to conduct activities in the Karamoja cluster side of Uganda with support from CEWARN Secretariat. 2. On Conclusions The DRAFT Report concludes that CEWARN is not administering a functioning regional Conflict Early Warning and Early Response System and must be regarded as stalled. We find this conclusion to be far￾fetched and untrue and at best very harsh indictment. It is true the situation room system is not yet operational but that does not mean that we have closed shop. In terms of expansion we are now operating in other parts of the country beyond Karamoja. The challenges that we may be facing with the move to the new initiative should not be viewed that we are not executing our mandate. We did talk about the low staffing level of the Unit but was not captured. F. Under recommendations: 10. External donors, including USAID, should use all means at their disposal to encourage member states to commit to attending at least one ordinary meeting of the Council of Ministers annually. - A more practical and implementable recommendation would be to lend support to the proposal by IGAD staff that IGAD to integrates and institutionalizes half yearly Council meetings and annual summit in the proposed revised and updated IGAD Agreement as is the case in SADC, ECOWAS and EAC. Please note that the push by development partners and IGAD for the updating of the Ernest and Young report has borne fruit and a consultant has been commissioned. 11. As a matter of urgency, IGAD should compile and publish the detailed progress made against its 2003 strategy, preferably before or concurrently with adoption of any new strategy. - This is an excellent addition to the earlier recommendations – it would be interesting to see how the USAID funding to IDDRSI PCU, ICPAC and CEWARN contributed to the overall performance of IGAD. G. Under CEWARN i. We would like to appreciate opportunity USAID gave to the evaluation team to review the draft report and incorporate our comments. While we acknowledge that there are several new revisions in the "final evaluation report," the report does not take into account many of the substantive comments we made earlier on--instead, it tries to justify or reemphasize the evaluators’ previous position as they appear in several sections. ii. Just to highlight one of the areas that remain problematic, the new system they note, "While the new system simplifies both incident and situation reporting, the evaluation team also independently identified a number of concerning insertions in the proposed new system that we believe will greatly complicate the conflict early warning and response system in a manner that will promote a theoretical approach, rather than local functionality (see Annex 11). It is also of concern that even the simplification of raw data collection under the proposed new system remains reliant on responses to subjective, ordinal (rating) scales, rather than on verifiable criteria." Unfortunately, the old system, which the evaluators seem to highly praise, worked in exactly the same way-- it relied on "subjective ratings" but apparently did a good job. What is majorly different with the new system is the expanded themes 162 and simplified data collection, which are some of the modifications done on the old system. iii. We strongly believe that the new system consolidates the old one and improves on it. Practically all parameters of the old system are captured in the new one. The additional themes were suggested by MSs, through a documented process that took over a year. And there are two dimensions to the system. The technical side, which we have been working on all through, and the user end. Once we are done with the technical, we intend to undertake training and start piloting the system in the second part of 2016. Only then can we be able to objectively assess the user end and respond to the argument that the new system "will promote a theoretical approach, rather than local functionality. iv. We would like to emphasize that the evaluation of the new system was based on second hand, and at best secondary information, and therefore failed to capture technical aspects of the new system to arrive at some of the conclusions. v. We wish to also reiterate that even after the meeting we held with CEWERU heads in March 2016, we did not get the impression of MS resistance to the new system as presented in the report. If anything, all the CEWERUs talked in support of the process, urging CEWARN to urgently get on with finalizing it, after the question was put to them. Indeed they agreed to have the system piloted in three MS: Somalia, Sudan and Uganda. vi. In conclusion, it is our considered view, and we would like to resign to the fact, that no amount of clarification we will give to make the report more objective will be taken in at this point in time. We therefore suggest that the evaluators submit the report as is, but clearly noting that CEWARN has a different written understanding. We propose that this understanding comprises a part of the Final report. vii. Finally, we wish to sincerely appreciate the evaluation for giving us an understanding of some of our blind spots especially in other areas of assessment. We will keep in mind some of the recommendations made by the evaluators as we go along with piloting and rolling out of the new system. 163