Revised Evaluation Report For The STRENGTHENING EMERGENCY RESPONSE CAPABILITY (SERA) PROJECT IN ETHIOPIA Prepared for: USAID –ETHIOPIA And Govt. of Ethiopia, DPPC By Shubh Kumar-Range and Teferi Bekele November 12, 2001 2 Abstract Ethiopia has experienced recurring drought, famine and other disasters such as epidemics, and massive population displacement. Over time, the problem of chronic vulnerability of the population has increased, as has dependence on food aid to deal with relief needs for large numbers of people facing severe food insecurity. Since the mid￾1990’s, the Govt. of Ethiopia has made a policy decision to invest development resources in disaster prevention and preparedness. Project Scope and Objectives In support of this effort in 1996, USAID signed a Limited Scope Grant Agreement with DPPC to strengthen the early warning system and enhance its efforts to better detect food shortages. In 1997, a Project Grant Agreement between the Government of Federal Democratic Republic of Ethiopia through DPPC and the United States of America was signed for the Strengthening Emergency Response Abilities Project (SERA), with the primary objective of developing vulnerability profiles so that effective planning and implementation of food security strategies could address the root causes of disaster vulnerability. Under the SERA project, an agreement was made for the four-year project with a budget of $3,700,000. The duration of the project was intended to be – from September 1997 – September 2001. There were three main elements of the SERA project – as laid out in the original project document of 1997. These were to: a) support the DPPC’s efforts to prepare vulnerability profiles in chronically vulnerable areas of the country; b) the development of response packages; c) conduct vulnerability studies/research (i.e. analysis of the root causes of vulnerability); and d) USAID Management Support. Past and other ongoing efforts on vulnerability in Ethiopia have tended to focus mainly on early warning and effectively reaching vulnerable population groups in emergencies. In contrast the focus of SERA was on disaster mitigation, and on effectively linking disaster relief with reducing chronic vulnerability through more effective disaster response and development. Implementation For implementing the project, DPPC engaged in a series of visits and consultations with Regions. In light of the decentralization initiatives ongoing in Ethiopia, and their relevance for building disaster mitigation, these consultations were instrumental in preparing the Revised Project Document by Federal DPPC in November 1998. This project represents the challenges of carrying out an innovative capacity building effort aimed at bridging the disaster relief and development divide. The effort was housed in DPPC – an organization known for its work in disaster response around the 3 country. An ambitious effort was launched with the SERA project that was aimed at providing: 1) An integrated, multi-sector approach using existing data and generate new information to develop useful Vulnerability Profiles and research on root causes of disaster vulnerability; 2) Generate a bottom up, participatory and decentralized effort that is run mainly by the Regions; 3) Capacity building through involvement of existing manpower in participating institutions. Scope of this Evaluation The Objectives of this technical evaluation were to: 1. Review and assess the progress of the project against targets and objectives set forth in the project agreement. 2. Assess problems faced in achieving results and make recommendations on how to improve project performance and implementation strategies in the future. 3. Based on the evaluation, make recommendations for completion of the original objectives, as well as critical follow-on activities in-line with deliverables as set out in the 1997 project agreement, over the period October 1, 2001 through September 30, 2003. The evaluation team reviewed project documents and met with project staff and stakeholders at Federal, Regional, Zonal and Wereda levels in each of the four regions in which the project is being carried out. This was done during a 3-week period in August 2001. An assessment of progress made and results was made from the perspectives of both efficiency with which the work was conducted, and the effectiveness of its outputs. Based on the objectives, the evaluation report considers in detail the following key outputs – institutionalization of SERA, capacity building, vulnerability profile development, research on root causes of disaster vulnerbility and development of response packages. Key Results and Observations This is a complex project in many ways. First, it has aimed at creating capacity to transition the formerly relief oriented DPPC towards disaster mitigation, and therefore to bridge the disaster-development divide. Second it undertook an ambitious and comprehensive data collection task at the wereda level – in many cases where little of no information was formerly available. Third, the concept of disaster risk and vulnerability is a complex, multifaceted one, and the project aimed to get a broad based consensus on the meaning and approaches to analysing it. And fourthly, project began with a well intentioned attempt to decentralize to regions the main task – the development of VPs – something that was not envisaged in the original project document. 4 There are a number of factors that have contributed to the low observed level of output to date, as compared with what had been anticipated or planned at the outset. These include a combination of management and technical issues that have been discussed in the sections relating to the five main outputs – Institutionalization of disaster mitigation work in DPPC structure, Vulnerability Profiles development, capacity building, indepth research, and Response Packages. The SERA VP development component has been confronted with start-up difficulties and delays that have had an impact on the timely achievement of results. In general, the activities planned by SERA for this component are progressing, but are significantly behind schedule. Given the methodological difficulties encountered in making aggregation and comparisons among the collected data and completing vulnerability analysis or research on root causes of vulnerability, it is uncertain if concrete results leading to recommendations and response packages can be completed by September 2001. As explained above, it is the opinion of the Evaluation Team that without having a clear analytical link with the goals and conceptual basis the VP process has been stymied. Expected results on Vulnerability Profiles, Research on Root Causes of Vulnerability, and Response Packages have not been achieved. The overall perception over time of the importance of bridging emergency response and development by developing effective disaster mitigation effort has grown over time, creating a greater demand for SERA results for mainstream planning and food security efforts. Lack of visible results has generated some impatience with its progress. This is understandable, given the urgency and need to respond to the chronic and acute vulnerability faced by large populations. Recommendations Recommendations for the next phase of this work for the period of October 2001 to September 2003 are based on the continuing rationale for, and much greater explicit recognition for disaster mitigation in Ethiopia, including in the present USAID-Ethiopia’s Strategic Objective. A high level of urgency should be given to producing quality outputs, especially the Vulnerability Profiles. Other elements such -- as the root causes research and response packages have floundered and also need to be reoriented somewhat so that progress on these important objectives can be made. Suggestions are also made for institutional integration, strengthening and capacity building in light of other related initiatives that are gaining momentum – such as the regional strategies for food security. 5 1. INTRODUCTION Ethiopia has experienced recurring drought, famine and other disasters such as epidemics, and massive population displacement. Over time, the problem of chronic vulnerability of the population has increased, as has dependence on food aid to deal with relief needs for large numbers of people facing severe food insecurity. Since the mid-1990’s, the Govt. of Ethiopia has made a policy decision to invest development resources in disaster prevention and preparedness. In 1995, the Relief and Rehabilitation Commission was converted into the Disaster Prevention and Preparedness Commission (DPPC). One of the primary functions of the new DPPC was - “to formulate strategies for future disaster prevention, undertake studies on causes of disasters, and set up systems for advance warning”. Even more significantly, the country’s National Policy on Disaster Prevention and Management (NPDPM), launched in October 1993 aims to strengthen development efforts in disaster prone areas by ensuring that “disaster prevention programmes shall be given due emphasis in all spheres of development endeavours”. The SERA project – to be supported by USAID and the Govt. of Ethiopia, and implemented by DPPC – was an outcome of this new policy emphasis. The NPDPM and other policies and programs, which aimed at promoting food security and tackling disaster, require accurate and timely information on the incidence prevalence, nature and causes of disaster if they have to be effective. Such information is critically important for both national and regional or local decision-makers to formulate and implement effective plans and programs and to target beneficiaries, in a way they can address the root causes of vulnerability, and enhance the food security strategy. In order to discharge its new responsibility of disaster prevention and preparedness, Disaster Prevention and Preparedness Commission (DPPC) made an appeal to donors for capacity building in 1996. In addition to this, a National Workshop on “Vulnerability in Ethiopia: From Disaster to Development was held from June 23 – 25, 1997. This was intended to help strengthen and enhance DPPC efforts in identifying contributors to disaster vulnerability, to better protect the population from food insecurities, and to channel emergency food aid and other assistance towards building resiliency and disaster mitigation. In support of this effort in 1996, USAID signed a Limited Scope Grant Agreement with DPPC, with two purposes in mind. One was to strengthen the early warning system and enhance its efforts to better detect food shortages and the second reason was to develop vulnerability profiles so that effective planning and implementation of food security strategies could address the root causes of vulnerability. Following the Ghion Workshop, a Project Grant Agreement between the Government of Federal Democratic Republic of Ethiopia through DPPC and the United States of America was signed for Strengthening emergency Response Abilities Project (SERA) on September 1997. This support was part of USAID’s Special Program Objective to Enhance Food Security in target areas of 6 Ethiopia (FY 1996-2001). Since then, the emphasis on strengthening the relief to development continuum has grown both within Govt. of Ethiopia, and for USAID, and disaster mitigation is now an integral part of the present USAID Integrated Strategic Plan for Ethiopia (FY 2001-2006). Under the SERA project, an agreement was made for the four-year project with a budget of $3,700,000. The duration of the project was intended to be – from September 1997 – September 2001. There were three main elements of the SERA project – as laid out in the original project document of 1997. 1) Support the DPPC’s efforts to prepare vulnerability profiles in chronically vulnerable areas of the country and the development of response packages; Developing a Vulnerability Profile (VP) for each of (up to) 16 zones. This activity will involve the preparation of vulnerability guidelines, mapping patterns of vulnerability, analysis of root causes and variables of vulnerability, and aggregation and dissemination of information collected. After VPs are completed, Response Packages (RP) will be developed – these are regional and zonal level contingency plans that detail appropriate interventions for particular disasters, populations and localities. 2) Conduct vulnerability studies/research (i.e. analysis of the root causes of vulnerability); Special studies, pilots and research to identify more effective ways of using food resources to target vulnerable populations and reduce the use of severe coping mechanisms – with the expectation that relief efforts are linked to development. A number of key issues for this research were identified for possible work to better understand the root causes of vulnerability – - Root causes of ecological degradation; - indicators of vulnerability, epidemics and coping strategies; - population pressure, carrying capacity of land and off-farm employment; - analysis of drought forecasting capability; and - targeting vulnerable populations. An objective of special studies would be to improve the ability of Title II food aid programming to enhance food security in vulnerable areas. 3) USAID Management Support: to be administered by USAID to achieve results in its Special Program Objective. 7 1. PROJECT RATIONALE, GOALS AND OBJECTIVES 1.1 Development of the Project Idea The foundation for the SERA Project was laid at a workshop on “Vulnerability in Ethiopia: From Disaster to Development” which was held at Ghion Hotel, Addis Ababa in June 23 – 25, 1997 (DPPC/PPPD, 1997). At the workshop the participants made recommendations related to Vulnerability profiles, In-depth research agendas and disaster response systems. One of the main recommendations was the need for development of vulnerability profiles that could identify the areas and population groups that are vulnerable to particular hazards/risks, and the reasons for such vulnerability. It was noted that previous efforts had been made with regard to vulnerability assessment in Ethiopia, including some vulnerability profiles of the drought prone areas and the establishment of the Early Warning System, which was accompanied by a variety of assessments and disaster monitoring. The previous vulnerability assessments and profiles included1: i) Vulnerability Profiles prepared in the 1980s in Wello (Hareide, 1988) and Hararghe (Humphreys, 1989); ii) The Profile of 5 Awrajas in Tigray, Amhara and SNNPR regions (Shawel-Consult International, 1993); iii) The EWS North Wello Survey (DPPC/EWS 1997). Vulnerability assessments have also regularly been made by WFP’s VAM and FEWS early warning system. In addition, NGOs in collaboration with DPPC, had also undertaken vulnerability assessments. These included, among others, the extensive SCF/UK Food Economy Study (Holt and Lawrence, 1993; Boudreau, 1998) and Nutritional Surveillance efforts (1993-1998), the CARE Food Information System (CARE, 1993), the CRS Growth Surveillance System (GSS), and the six USAID/NGO partners’ Food Security and Nutritional Baseline Survey (CRS, 1997). This information had been used to supplement the country’s Early Warning System (EWS) which was started in 1961. This EWS effort focused mainly on the assessment of the relevant disaster signals in different parts of the country particularly on transitory food insecurity, and was primarily geared to locating where additional food aid should be targeted. The main assessments have included the pre-harvest and post harvest assessments in the crop dependent areas, pastoral areas assessment and climate/disaster monitoring. However, the existing information, which is generated by EWS and from small-scale studies, is inadequate for identifying specific characteristics of at-risk populations, causes of vulnerability, or needs, options and opportunities for undertaking prevention measures. Because of these limitations, DPPC and Regional Disaster Prevention and Preparedness Bureaus lacked sufficient information on vulnerability to 1 These reports are included in the Annotated Bibliography on Vulnerability Assessment produced for the Ghion Workshop of 1997. 8 famine and other disasters to plan prevention and mitigation efforts, and to assist other sectors in addressing these needs. In summary, the previous studies and profiles were found to be limited in their scope to address the interrelated vulnerability factors. Moreover, they tended to focus more on transitory food insecurity, without addressing the processes of vulnerability. This was therefore, the justification for the SERA project. It was hoped that the development of the vulnerability profiles of the drought prone areas and an in-depth research on vulnerability to disasters in general, would help to understand the root causes of various disasters, to devise response packages and to target vulnerable groups/areas for disaster prevention and preparedness. 1.2 Further Development and Revision of Project Concept The 1998 Revised Project Document Based on a series of visits and consultations at Regions, a Revised Project Document was prepared by Federal DPPC in November 1998 and discussed in a one-day Discussion Forum with Regional DPPC offices to finalize its contents. This revised document expands the original project concept into more concrete operational guidelines, and administrative procedures for implementation of this complex, multi-level program of work all the way from Wereda, to Federal levels. In particular, a division of responsibility between the Federal and Regional levels for different aspects appeared to have been sought. Substantively, it also identified 3 major products for SERA, though in a slightly different configuration from the original project document. The 3 outputs that were identified in the 1998 Revised Project Document were: 1) Relevant and useful Vulnerability Profiles – Regions with the support of F/DPPC “will develop vulnerability profiles by zones and Weredas” for selected disaster prone areas of the country. This document identifies a total of 18 zones and 80 Waredas as the most vulnerable, and therefore the intended targets of this project. 2) Applied research and in-depth studies on root causes of vulnerability – “DPPC through its policy, plan and program department has laid the groundwork to coordinate vulnerability research by lead agencies at federal level”. The five broad areas identified in the Ghion Workshop, and adopted in both the 1997 and 1998 project documents were: -Root causes of ecological degradation -Indicators of vulnerability, epidemics and coping strategies -Population pressure, carrying capacity of land and off farm employment -Analysis of drought forecasting capability -Targeting vulnerable groups 9 3) Strengthened response packages – or response mechanisms and development interventions through incorporation of the results of vulnerability profiles and research results. In addition this document identifies “Bottom – up Response Packages” will be tried out in selected areas. The original project document of 1997 elaborated on the content of Vulnerability Profiles, and proposed a series of well defined outputs such as – guidelines for vulnerability profiles, vulnerability factor analysis (to assist in predicting increased risk in areas), historical hazard maps, mapping patterns of vulnerability, analysis of root causes and patterns of vulnerability, and response packages or contingency plans for vulnerable areas. No such detail on content of VPs is given in the Revised Project Document of 1998. 1.3 Approach for meeting project activities The 1998 Revised Project Document identified three major elements of the overall approach that was to be used: 4) An integrated, multi-sector approach using existing data 5) A bottom up, participatory and decentralized effort 6) Capacity building through involvement of existing manpower in participating institutions. 1.4 Project Scope and Implementation Process Beneficiaries: Regions, zones and Weredas, and “all the most vulnerable communities in the targeted areas.” The project is stated to cover four regional states and their most highly vulnerable zones. The target zones identified were: Tigray (Southern, Eastern and Central zones) Amhara (Wag Hamra, South Wello, North Wello, North Gonder, South Gonder zones ) Oromia (Eastern Hararge, Western Hararge, Borena, Bale and Arsi zones) SNNPR (North Omo, South Omo, Kembata, Alaba and Timbaro, Hadiya zones) All Weredas in these zones were to be targeted for development of Vulnerability Profiles. Phasing: During the pilot phase – the first field operational year, two zones per Region would be selected for capacity building and “field testing of primary data collection and analysis in limited areas” prior to expansion to all sites. This pilot phase was intended to be implemented in 1999. Activities at Different Levels: The 1998 project document includes a detailed set of activities at each of Federal, Regional, Zonal and Wereda levels. 10 At the central level, Federal DPPC and SERA staff responsibilities were to provide technical guidance and support for the major products, as well as administrative and organizational support to regions. In addition, the in-depth research component was identified as a Federal role … “DPPC through its policy, plan and program department has laid the groundwork to coordinate vulnerability research by lead agencies at federal level.” (Revised Project Document, 1998). The main Regional responsibilities identified in the 1998 document were: to provide comments and suggestions to the Federal initiatives; to participate in the various workshops organized for methodological or other trainings by Federal DPPC and SERA; to set up, support and manage the Regional implementation units and interdepartmental coordinating committees at all levels; and to compile Vulnerability Profiles for Weredas, zones and Weredas. At Zonal level, setting up, supporting and managing implementatation units and coordinating committees, compiling and analysing Wereda vulnerability information to be collected, and its dissemination. At the Wereda level activities included – review of disaster experiences and data collection from secondary and primary sources. 1.5 Pilot Phase and Expansion to Scale During the first year of the operational project, an initial set of 8 target zones (2 from each of the four regions participating in the project) and 16 Weredas were planned to be covered for the VP development. After that, during the second and third years of operation, the project was to have expanded to approximately 18 zones/special weredas and 80 weredas – a five fold increase in the volume of work. It was expected that the pilot phase would receive a major investment in capacity building, and it would be relatively easy to expand from that foundation. 2.6 Institutional Arrangements for Project management At the Federal Level, SERA project is housed in the Policy, Planning and Programming Department (PPPD) of DPPC. Staff include SERA National Coordinator and other SERA staff, DPPC supporting staff from its PPPD and staff from Birhan Constultancy. Together they constitute the Federal VIMU, which is headed by the Director of PPPD. Vulnerability Information Management Units (VIMU) were to be set up at Regional and zonal levels in the implementing agencies – initially expected to be the DPPB or DPPDs. Within the VIMUs it was anticipated that SERA and assigned agency staff would together take on all aspects of project implementation. VIMU members would be a combination of SERA project staff and staff of implementing agencies, and would be headed by the Regional or Zonal heads of the implementing agency. 11 Vulnerability Assessment Group (VAG) at National level and Vulnerability Working Groups (VWG) at Regional and zonal levels were intended to provide technical oversight and assistance for project implementation. VAG and VWG members would be drawn from a variety of related sectors such as MEDAC, Agriculture, Environment Protection, Health, as well as from donor agencies, NGOs and universities with substantive expertise and interest in the subject matter. Regular meetings of these technical groups were expected to provide ongoing technical input and support for implementing the SERA project. Within DPPC, an integration of this work with that of the Early Warning Department was foreseen. Between sectors, a close cooperation between this work by DPPC and MEDAC was to be encouraged. At the Wereda level, the Wereda Council was to oversee project implementation. The project staff would head and comprise the Data Collection Group (DCG). This would be supported by a VWG whose members would be drawn from Wereda sectoral offices and who would assist in data collection and other project support. 3 EVALUATION METHODOLOGY 3.1 Terms of Reference The Objectives of this technical evaluation were to: 4. Review and assess the progress of the project against targets and objectives set forth in the project agreement. 5. Assess problems faced in achieving results and make recommendations on how to improve project performance and implementation strategies in the future. 6. Based on the evaluation, make recommendations for completion of the original objectives, as well as critical follow-on activities in-line with deliverables as set out in the 1997 project agreement, over the period October 1, 2001 through September 30, 2003. 3.2 Overall Approach and Issues Addressed The original project document signed in September 1997 was for a four year project, with a completion date of September 2001. However progress has been significantly delayed due to the complexities of implementation of this multi-level and multi-purpose project, as well as the growing importance of decentralization, need for Regional ownership and capacity building. Added to this were substantial technical and coordinating problems, and as a result, the work is still in its pilot phase at this time. As a consequence of this delay in project execution, though the timing of this evaluation coincides with a 12 completion or ex-post evaluation, it is intended to serve the purpose of a mid-term evaluation, ie., to provide insights for improved project performance in future. The evaluation was therefore designed to: 1) measure progress towards meeting objectives, 2) analyse the factors that helped or hindered this achievement, and 3) to assist in supporting performance of implementing agencies. For each of the major outputs expected for the project, information was sought on: -progress towards objectives, -efficiency of the management and implementation process; -effectiveness of product delivery – quality, value, utilization and impacts -constraints -recommendations In order to be consistent with accepted evaluation procedures, the focus was on transparency, accountability and performance through analysis of all available data using a variety of sources, and incorporating knowledge, perspectives and values of all stakeholders. 6.3 Work Plan and Methods Project Briefing: Detailed briefing provided by Federal VIMU staff, and USAID-Ethiopia FHA Office. Additional details and information provided by Project Coordinator. File Project Review: Reviewed all project outputs and progress reports, as well as technical and methodological material prepared for project implementation. Preparation of Workplan: An evaluation workplan was prepared which included: 1) An overview of the project giving a summary of project background, context, and goals 2) Evaluation mandate, including reasons for the evaluation, its scope and focus, and a clarification of objectives 3) Evaluation matrix, including a review of project rationale, its most critical deliverables, progress on critical deliverables, changing policy context for disaster mitigation 4) Methodology, including sources of data/information and procedures 5) Schedule of activities 6) Report outline 13 Development of Instruments: Interview checklists were developed for SERA/VIMU staff, and another for stakeholders. Checklist for SERA/VIMU staff at Federal, Regions, Zones and Weredas: - Progress: methods, outputs, results - Input by stakeholders on technical aspects of methods used, and project implementation - Institutional integration and sustainability - Utilization of outputs/results to date - Future planned outputs, dissemination, applications - Constraints and needs Checklist for Stakeholders at Federal, Regions, Zones and Weredas: - Input in SERA process, methods - Expectations, awareness of outputs, uses – actual and planned - Issues, concerns and recommendations for completion of Pilot Phase, utilization of results and for Expansion Phase Data Collection: Sources of data were all available and pertinent project documents, and stakeholder interviews. Interviews were conducted with key SERA personnel at Federal and Regional level (in all four project Regions), Zonal level (from all eight zones covered by project) and Wereda level (where available2 ). Interviews were also conducted with VIMU staff from host institutions, including the head of VIMU units at Federal, all four regions, and selected zones in each of the four regions. Finally VAG (at federal) and VWG members at Regional, zonal and wereda levels were interviewed. Data Analysis and Report Writing: An initial draft report is planned for September 21, which will be reviewed by USAID￾Ethiopia and DPPC at Federal and Regional levels. Comments provided will be incorporated by the Evaluation team, and a final report presented to USAID-Ethiopia and DPPC. 4 RESULTS, CONSTRAINTS AND RECOMMENDATIONS FOR KEY PROJECT DELIVERABLES 2 Wereda level SERA staff were no longer present since the completion of data collection activities at Wereda level. 14 4.1 Institutionalization 4.1.1 Regional Ownership Based on the June, 1997 Ghion Workshop recommendations and September 1997 project grant agreement with USAID, DPPC/F carried out consultations with Regional authorities on the desirability, concepts and approach for project implementation. Given the decentralization process underway in the country, it was determined that Regional ownership of activities was an essential element for the success of the effort. Similarly, at the Regional level, the process of involvement of Zonal and Wareda governments and departments was undertaken. This initial process took about one year, and the Revised Project Document was finalized in November 1998, following its endorsement and acceptance by the Regions, and respective zones and Weredas. Following the endorsement of the project by regional DPPC/B/Ds, DPPC/F/SERA proceeded with facilitating the selection of focal point institutions at the Regional, zonal and Wareda levels. 4.1.2 Regional integration in implementing mechanisms In order to generate the multi-sectoral stake-holder participation in development and application of the vulnerability profiles, it was sought to involve the active participation of professional experts from different disciplines. To this effect, the project has established a couple of mechanisms to co-ordinate different government and non￾government organizations. These are: 1) Vulnerability Information and Management Unit (VIMU) – this is the operational and implementing arm, and is primarily composed of SERA employees, supplemented occasionally by hosting institution staff. 2) Vulnerability Working Group (VWG) – this is intended to be the technical support and consultative ad-hoc group to be convened by the hosting institution to provide ongoing substantive input for project implementation. Both VIMU and VWG were expected to operate in some form at regional, zonal and wereda levels. In the National guidelines prepared at the Federal DPPC methods of establishing VIMU and VWG at all level and the roles and responsibilities of the participating individuals and organizations are defined. Based on this, all project regions have established the VIMU and VWG. At the Federal level, DPPC acts as executing agency and has housed the SERA project. At the Federal level, a VAG had been formed earlier to support the country’s early warning activities, and its role of providing technical support was extended to cover this activity as well. Its composition which included Govt. departments, international agencies and NGOs, was intended to facilitate efforts of inter-sectoral coordination, provision of technical assistance, to assist the implementation process and to provide the necessary guidance for implementers, etc. At the regional and zonal levels VWGs compose of experts drawn from collaborating bureaus and line departments, were established to ensure the participation of various 15 sectors in VP development. The members of the VWG are technical experts who work on the technical aspects of the VP. In addition, VIMUs have been established in each of the project executing bureaus or offices of the regions and zones to coordinate and organize various activities and to monitor the project progress. There is some variation in the institutional set-up of SERA by region. In the SNNPR, the SERA regional project office is housed in the Disaster Prevention and Preparedness Office (DPPO) and in the Bureau of Planning and Economic Development (BoPED) at the zonal level (as there are no DPPO suboffices at zonal levels in this region). At the regional and zonal levels VWGs composed of experts drawn from collaborating bureaus, regional VIMU in the DPPO and zonal VIMUs in the respective department of PED have also been established. In the Tigray Region the SERA project office is housed in the Food Security Desk of the Regional Council. As at the regional level, the zonal VIMU and wereda DCG are also housed in the Food Security offices. The regional and zonal VWGs, composed of experts drawn from collaborating bureaus are established. This region has established an additional Technical Task Force to help overcome weaknesses in the effectiveness of the VWG mechanism (discussed later). Individuals in the task force are drawn from a combination of Regional NGO, University, private sector, and government departments, and are all contributing substantively in the technical analysis of Vulnerability Profiles. Both in Amhara and Oromiya Regions, regional and zonal SERA project offices are housed in the respective Disaster Prevention and Preparedness Commission/Bureaus. In both regions VIMUs and VWGs are accordingly established at regional and zonal levels. The Wereda VWGs comprised of experts from Wereda sectoral offices (agricultural, health, education, NGOs if any, etc) were formed in all the target weredas. In all Weredas DCGs were formed through drawing experts from focal point offices and headed by Wereda technical coordinators. DCGs have participated in the collection of secondary data and supervision of the primary data collection in the respective Wereda. 4.1.3 Administering Regional SERA There is a complex division of responsibility for administering Regional activities, with the Federal SERA and DPPC offices not only involved in procurement of equipment etc. for regions, but also in financial resource allocation between the regions. It was found that though there is a clear division of responsibilities and activities between federal and regional administering institutions, no such division of the budget is made. As a result Regions have no assigned budget, but have to make quarterly activity and budget requests, with no assurance of what will be forthcoming. An important objective has been to fully integrate SERA in DPPC procedures for all management functions. As a result existing rules and procedures in effect are being used for all procurement and recruitment needs. Given the decentralized nature of this project, an over reliance on a centralized model of financial and administrative management, may 16 have been contradictory to the objectives of regional ownership, and not fully supported the efficiency and effectiveness of project outcomes. 4.1.4 Efficiency and Effectiveness There is a high level of regional ownership and acceptance of the SERA concept and functions. In all the targeted regions, members of regional and zonal VIMUs are meeting regularly to accomplish the objectives of the project. The Zonal VIMUs have participated in the collection of both primary and secondary data, as well as in data processing and analysis. Regional VWGs members at all levels have assisted SERA in the collection of both secondary and primary data for the Pilot Phase weredas. Though there have been relatively better co-operation and co-ordination among the respective VIMUs and VWG members particularly during the training and data collection activities, this was not sustained in the next stage of the VP development activities. The Regional and zonal VWGs have not had any regular meetings and practically not been functional, since the completion of the data collection. During the discussions with the evaluation team nearly all of the VWG members interviewed expressed dissatisfaction with regard to co-operation and co-ordination process. Discussion, for example, revealed that in all regions, the project has not been able to organize any formal meetings for VWG members for the last 18 months and as result participation of the VWG members in the development of the VPs has been inadequate since the completion of data collection. Problems were also cited with staff turnover in participating institutions, and hence the absence of continuity and therefore building of ownership. Work pressures, and SERA participation being seen as taking on an extra workload, ie. Not part of job description or key responsibilities was often a hinderance in ensuring the substantive inter-sectoral input and participation envisaged for the VWG. At the same time, some members interviewed by the Evaluation Team emphasized the need for an attitudinal shift and an increase in confidence and feeling of ownership when talking about vulnerability assessment. In order to fill the gap created as a result of the weakness of the official SERA technical support mechanisms, an additional group has been informally estabalished in Tigray. Here, a Regional Steering Committee provides technical expertise from academic circles like the University of Mekele, Relief Society of Tigray (REST) a local NGO, etc. have participated. The regional steering committee in Tigray making division of tasks among members depending on their specialization and capacity has shown improved performance in the effort to develop analytical model for the already collected vulnerability assessment data. As a basis for promoting active participation of the VWGs in the whole process of the project implementation, SERA should have focussed on awareness creation to build up the required conceptual understanding of Vulnerability, and the substantive links to the 17 intersectoral development agendas of its members. Instead the focus has been overly on data collection needs, and as a result turning VWG into an arm of the SERA VIMU functions. Effectiveness of VIMUs have been hurt by staff turnover both in SERA, and in hosting institutions at regional and zonal level. This has become a problem at the stage of data analysis. The decision made to house SERA offices in the DPPC/Bs at federal, regional and Zonal levels (except in Tigray and zonal level of SNNPR), is likely to have been a correct decision, both from the perspective of administrative convenience and the potential for capacity building on issues of vulnerability. However, shortage of adequate technical personnel of the required qualification in regional DPPC/B/D as well as generally a poorly linked institutional experience with developmental sectors, has placed them at a disadvantage. The Amhara DPPC, may be the only exception to this, and has both the ability and commitment to integrate its functions with mainstream development efforts. The relative strength of this office puts it in a better position to develop the products, and integrate them in both the early warning/relief oriented work, as well as in food security and overall planning processes. In Tigray, the mainstreaming of this project with the high profile Food Security Desk activities, offers a good opportunity for both sustainability and effectiveness of providing an input into the planning processes. However, the DPPB and early warning links here have weakened. In SNNPR, though the DPPD office has a low profile, and the capacity here is comparatively weak, there is a lot of interest in this work in the Regional and Zonal councils. This could provide new opportunities for utilization of results, including in cooperation with the new food security strategy work underway. In Oromia, time limitations and staff unavailability limited the opportunity for the evaluators to interview stakeholders and VWG members. However, given the large number of highly disaster prone weredas and zones in this region, we feel that the inability to fully address the needs here is a major gap in this report. In terms of the effectiveness and sustainability of SERA efforts, the capacity and development integration of the hosting institution is clearly an important factor. In Tigray, for example, SERA staff and their results are strategically positioned to assist at the development efforts of the regions Food Security Desks at all levels as part of their daily routine. Because of this integration, the Tigray experience of housing SERA in the Food Security Desks also adds a strong element of accountability – given the high profile nature of their responsibilities. This particular area of sustainability will become increasingly important for SERA, as the project support could terminate sometime in future. If the purpose of strengthening the connection between relief and development is to be met, and DPPC’s ability to address disaster mitigation improved, a serious reconsideration is needed on the most effective locations and institutional arrangements for SERA capacity building so that an integration of relief and development can best be made. The project organizational structure prepared for regions seems complicated and didn’t show the existing conditions of the regions and clear line of responsibilities of each organizational unit. According to the Organo gram distributed during workshop 18 (19/03/1998) each unit has only a technical responsibility with out any clear accountability to each other. Most of the members of the VWGs interviewed are aware of the importance of the VP information and feel the responsibilities indicated in the guideline is within their mandate, but are skeptical to fully participate in the project for the following reasons; q Since the responsibility of implementing the project and terms of financial flows, the established VIMUs, the monitoring of technical, administrative and financial activities and etc. lies wholly on the RDPPC/Bs. q Though for the most part, there was a lot of support for the role of DPPC to help to link up relief and development work, a few of the VWG members interviewed did not see the role of DPPC/Bs as the appropriate institution to coordinate such project and program. It is possible that the gradual shift of SERA activities from a disaster vulnerability focus to a general poverty/food insecurity analysis, has contributed to this view. This perception may also be reinforced by their past experiences, and current technical expertise and programming involvement of DPPC/Bs. SERA as an institutional strengthening project, must find ways to enable its partners to take action to eliminate or counteract the negative impact of factors such as high staff turnover on training progress and sustainability of outputs. Successful coordination and partnership building is one of the main challenges faced by DPPC/SERA in developing the VPs. Some improved cooperation by VWG members has been observed during the early stage of data collection, but this didn’t extend to the next stage of the VPs development. Ex-post it is not possible to fully understand all the causes for this – however it appears that the input by this group in the process of development of Vulnerability profiles or in their application was not fully considered or perceived to be necessary/desirable. Several VWG members interviewed in the Regions, and VAG members interviewed at Federal level expressed disappointment at not being involved and continuously informed of the plans and progress of SERA. VIMUS are expected to function as a permanent organ RDPPC/B or PED to continuously prepare VPs even after the termination of the SERA project. However there is no clearly defined and established strategies as to how this is possible. There is also lack of clarity at the time of this Evaluation, on the final content of VP and their applications in responding to disaster vulnerability needs. The already existing Early Warning Department in the federal DPPC and Units in the RDPPC/B are working independently from the VIMU. Making a substantive connection between the work of SERA and the EW Units would be good start for DPPC to provide a solid justification for the value of SERA continuation. The ability of SERA outputs to make a contribution to other sectors, and NGO efforts in disaster prone areas is another indicator of success. Very little attempt appears to have been made to use expertise from Ethiopian academic circles to provide technical input, except in the case of Tigray where the observed limitations of VWG input have been supplemented by a small expert group drawn from a combination of sources (inlcuding, University, NGO, Govt. departments). A high 19 reliance on the VAG and VWG as a sole mechanism for technical input is expected to have contributed to a weak technical accountability system in the project. This may also have limited the ability of SERA to utilize VWGs as a stakeholder forum in the process of developing vulnerability profiles. 4.1.4 Constraints and Recommendations The main constraints were: v Unavailability of regional budgets and cumbersome reporting procedures via Federal DPPC have constrained the ability of regions to budget, plan and implement activities. Some improvements have been put in place recently, but they do not appear to be enough. v Cumbersome financial reporting mechanisms in which federal DPPC procedures impede a timely and predictable funds release process from USAID. v Difficulties in getting a timely reporting of expenses from Regional SERA offices v Insufficient flexibility in SERA staff recruitment procedures to be able to provide incentives for retaining staff v Variable ability of regional institutions, especially DPPC to fully integrate SERA into their structures and activities, and to allocate permanent staff to VIMUs v Overemphasis on VWGs as a data collection arm for SERA, and insufficient utilization as stakeholder and users of VPs The main recommendations for future activities are: v Establish regional budgets to match workplans for decentralized capacity building initiatives such as SERA. This will provide a more flexible and empowering context for regional teams. This will however, also require attention to supporting management capacity at regional level (where needed), to enable to enable sound financial reporting and related management systems and practices. v Establish clearly the appropriate institutional capacity building needs for Disaster risk reduction research (eg. Vulnerability Profile and associated research) and its application for both development planning and emergency response. In the case of SERA, the emerging decentralization in the country during the project start up period led to the appropriate judgement to build a decentralized project. Since then, the overall development planning context at Regions, including the Food Security Strategy work has become much better established. This creates a whole new set of opportunities for disaster mitigation work, that were not available at the start of SERA in 1997. At the same time the involvement of DPPC and its Regional bureaus should also be considered important – to link work on disaster mitigation with its Early Warning and disaster response. v An effort should be made to build on the institutional capacity building carried out in SERA. v The experience in SERA of the problems with intersectoral advisory committees to either provide technical support and guidance (VAG, VWG) or carry out the actual activities (VIMU) has shown that such adhoc approaches do not provide do not provide adequate institutional commitment or input for sound functioning. A more 20 realistic approach should be taken, given the heavy work commitments of other bureaus. Several options should be considered, and if possible the ability to draw on input on an as needed basis may be considered, with a budget explicitly available for this function. 4.2 Vulnerability Profiles 4.2.1 National Guidelines for Vulnerability Profile Development The concept of vulnerability is not a straightforward and vulnerability assessment is a complex process that involves multi-levels and multidimensional factors. Given the complex nature of vulnerability profile development, limited experience in Regions and lack of trained manpower, the preparation of the National Guidelines for Vulnerability Profile Development by Federal DPPC/SERA was an important first step for effective implementation of the project. In line with this understanding, the national guidelines for VP development were prepared to provide overall guidance and consistency in the process of data collection, analysis and VP development at all levels A National Guidelines for Vulnerability Profile Development were prepared by Federal DPPC/SERA and presented to Regional representatives and other professionals in August 1999 in a training workshop. Prior to that, in May 1999, a technical review was carried out by presenting the draft guidelines to the VAG at Federal level. Extensive feedback was received, and has been reviewed by this evaluation team. The draft guideline was then distributed to regions for their comments. After consolidating and incorporating all the comments that came from regions and various stakeholders, a national workshop was held to finalize the guideline3. In the national workshop, regions, donors, NGOs, VAG members and pertinent organizations participated and made their final comments. After the workshop, the Federal DPPC produced the final version of the guideline and distributed to all users, particularly to regions. The national guideline has been used by Federal DPPC/SERA project team, Regional DPPBs/C and VIMU members as an operational guide at each level for the development of VP in the pilot phase. The main purpose of these guidelines is to assist the process of data collection for development of vulnerability profile. The guideline includes a list of detailed indicators and methods of data collection to provide a fairly uniform and consistent instruments and information for vulnerability profile development. Following these guidelines, the project has attempted to collect and analyse the following types of information: Secondary data collection (SDC) at wereda, zonal and regional levels; wereda level key informant 3 Though this may have been the intent, we were informed in regions that there was very little time for them to provide feedback. Many believed that they would have an opportunity at the National Workshop to discuss it, but found it to be more of a training oriented forum, with the content of Guidelines not subjected to any serious discussion of any reservations they may have had. 21 interviews; wereda level focus group discussions; wereda level representative household surveys (HHS); and wereda level representative women and children individual surveys. 4.2.2 Implementation of Vulnerability Profile Development Data collection instruments were developed and pre tested in one Wereda, having three different Argo ecological agro ecological zones (i.e. Kolla, Woyna Dega, and Dega) in north Shewa. The purpose of pre testing using AEZ was to be able to incorporate varying conditions in the three major agroecological zones into which areas of Ethiopia are usually categorized. Based on the pre test finding and comments provided from regions, concerned line departments & NGOs, modifications were made on the data collection instruments – with some variations by region made in the local level adaptations made later. The main phase of data collection was January to March 2000. Altogether, primary data was collected from 93 sampled Peasant Associations (PAs) of the sixteen target Weredas in which 9,794 household were interviewed. In addition to the household survey data, information was also gathered from key informants and community leaders as part of primary data collection. Accordingly, a total of 1,317 key informants were involved in the in-depth interviews in all peasant association. Existing records and other quantitative sources of information and maps were used as main secondary data sources. Secondary data tabulation sheets were used to solicit wereda and zonal officials for existing information. In the majority of cases, much of this information is not compiled for statistical purposes, and therefore information availability was patchy. Information that was available, was reported of poor quality and /or and not available for longer time series to use for purposes of VP development. The project should have realized these problems associated with secondary data and decided to use shorter time series instead. Accordingly, many of the indicators of the secondary data were found quite unreliable for purposes of vulnerability trend analysis, and the original list of indicators was reduced to 66. 4.2.3 Outputs A large amount of information is now available from the 16 pilot weredas from a number of sources. This information can provide a variety of useful background and baseline information on the Weredas. In many cases, this may be the only source of representative Wereda level information on living conditions available. As a result, it is likely to be a useful resource for many different users. There is however, a great deal of uncertainty as to how this voluminous information can be most usefully compiled for general use, let alone for the purpose of developing useful Vulnerability Profiles. Upto the time of this evaluation, a great deal of tabulations have been accomplished from the various information sources, and indicators. However, there is so far no integration of different information sources, to answer the key questions posed as part of the conceptual basis of Vulnerability Profile development. 22 4.2.4 Conceptual and Analytical Framework for Vulnerability Profile developed by SERA The evaluation of SERA finds that the project leaders attempted to create a broad based understanding of the concepts of vulnerability. This is reflected in the National Guidelines. This is a commendable effort, and is a valuable component of capacity building. There was however, a need early on, to establish a more concrete analytical approach that would be used. Instead, it is the understanding of the evaluation team, that it was expected that each Regional Team would take the general conceptual ideas presented and translate them into specific analytical methodologies, geared to the disciplinary background and interests of the Regional Teams. The assumption that the Regional Teams would be able to take the general principles and apply the broad based data collection into concrete may have been implicit in the overall approach taken. This was to some extent implied by the input given to us by the SERA coordinator. Vulnerability analysis can be done in many different ways. Many of the methods used are data intensive (eg. CART) or are suitable only for early warning type of responses (GIEWS and Food Economy approachs). It is however indisputable that vulnerability assessments must be methodologically sound and context specific. Therefore it should be considered important to decide which is the best and most suited approach for Ethiopia. A broad based understanding is useful for this, but the application of this knowledge for delivering a useful set of results is equally important. This latter effort needs to be sharpened considerably. It appears that a consensus-based approach has been used for creating the foundations on which Vulnerability Profile development has been undertaken by SERA – ie., relying on a broad base of disciplinary and technical expertise to contribute. While this is commendable, such an approach is unlikely to yield a clear and focused result. Collecting a broad base of data and information – to allow various disciplinary approaches to be taken -- appears to have been an objective that resulted from this approach to methodology development, and the lack of a clearly defined analytical approach.. However, none of the experts who were consulted in the development of the long list of indicators were involved in the data processing, analysis or discussion of the application of Vulnerability Profiles. As a result, regional teams were on their own, with some input from Federal SERA (this was often usually late – eg. Data entry training about 4 months after completion of data collection, and data integration and analysis training a year after completion of data collection). This is another example of a disconnect between the ideas and their application in the SERA project, which have contributed to the observed results. Past vulnerability profiling work in Ethiopia has typically focussed on early warning and improving ability to respond to crises. The orientation of SERA was different – it addresses the issue of chronic vulnerability to disasters, and to making the link with development processes. The link of this effort therefore, with both the Early warning work in Ethiopia as well as with the development planning and food policy would have 23 been desirable. The conceptualization of vulnerability to be able to make these connections may be considered in future. There is another crucial aspect of vulnerability that should be incorporated – what are the key elements that need to be monitored over time? What links can be made with the ongoing Early Warning data efforts, and what supplementary data needs have to be considered once SERA is completed? Even getting to these questions will need that a clearly identified approach for assessing vulnerability and hazard risk. There is a range of quantitative and qualitative methods that can be used, and an appropriate choice is required. It is likely that qualitative approaches are better suited to enabling greater weight to the participatory elements of vulnerability assessment, and also to reduce data collection and management. The right balance for meeting the specific needs in Ethiopia, in a practical and doable way is required. It is our ex-post assessment that establishing a clear link between the needs of the Vulnerability Profiles with data collection and analytical framework from the outset would have been useful. It is necessary to link the broad based vulnerability concepts and the clear set of questions that were identified early on in the SERA project to the actual data needs and its analysis. In the National Guidelines, we find a disconnect between the questions posed for Vulnerability analysis (who, where, when, why, how and what is the nature of hazards, risks and coping strategies), and the conceptual framework which identifies ‘Causes of Chronic Vulnerability’. The Causes of Vulnerability are then translated into ‘Indicators’ in the six content areas identified. Is vulnerability to all hazards equally distributed? Or does risk to different hazards vary within the population. This lack of clarity has spilled over into the Vulnerability Profiles – in which the entire description of the areas is presented in terms of Indicators of Vulnerability. The reader is unsure from one section to the next what is the interpretation in terms of the Vulnerability of the population. There is no guidance on how this large set of indicators (causes?) of vulnerability, are to be used answer the seven key questions posed. How is vulnerability and risk to different hazards to be assessed? It is our recommendation that a clear link between the objectives of Vulnerability Profiles with data needs and an analytical approach from the outset would have facilitated the data collection, analysis and reporting of Vulnerability. The material, for example, that was presented in the March 2001 SERA workshop on data analysis, is a reflection of an ex-post look at what to do with data that has been collected. This is a dizzying array of vast amounts of information that is at best linked to some broad concepts, with no ideas on interpretation or analysis. Where is the information leading to? How will it be used? We have a long list of indicators: · Demographic indicators – 23 in all at least · Sociocultural indicators – 23 approx. · Indicators of coping strategies – 11 · Physical and climatic indicators – 24 · Land use and holding indicators – 6 · Indicators of Agricultural production, pests and diseases – 26 · Income, employment and wealth – 8 24 · Food security – 12 · Social and related services – 24 · Agricultural and related programs – 18 · Risk reduction institutions – 7 · Human health and nutrition – 25 approx. · Women’s and children’s nutrition – 5 approx. · Disaster History – 8 approx. Many of these were obtained from more than one data source. It was therefore surprising to find that Vulnerability Profiles chapters were written not by data source and not by topic. The type of information that was available in the approx. eight Vulnerability Profiles that were ready in partial form at the time of this evaluation – can be considered to be useful more for purposes of Data Documentation purposes, rather than for analytical purposes. In using this approach, great deal of work still remains for completing the analysis – ie. First converting the different data sources into one cohesive discussion of various ‘indicators’ and then to translate the information into the key questions posed at the outset – who, where, etc. 4.2.5 Assessment of Methods, Quality and Relevance of Data Issues: · Methods of data collection and cleaning A great deal of effort has been made by SERA to develop the large set of data collection tools, pretest, train data collectors, supervise and check data, etc. Not only is the list of ‘indicators’ large and comprehensive, but a variety of data sources have been used for many of them – eg. Both primary and secondary sources of data were collected for most indicator types, and typically the same indicator was covered in the three main primary data sources – questionnaires, key informant interviews, and community group discussions. There is no question that a more focussed effort would be much more cost-effective. The large amounts of information, also required a large data entry and cleaning effort. It is not clear why there was an approx. 6 month delay from the time the data collection was started, to when the data entry was commenced for the household/womens/children survey information. It is possible that the teams were fully engaged in data collection – and that a parallel data entry would place an additional strain on the staff. There may also have been other constraints, such as the availability of computers. Data entry can be facilitated with use of precoded questionnaires (coding information can typically be obtained from pretests), however given the decentralized and multi-focal set up for data entry could have created some complexity in setting up relatively uniform set of codes. Some data cleaning was conducted as part of criteria for acceptable ranges that was set up in data entry formats. It is not clear if any additional data cleaning was 25 conducted. Queries to this effect, to the regional staff engaged in data analysis, suggested that no additional data cleaning was done. Information was also not available on how ‘missing data’ problems were handled, and what percentage of cases were dropped from analysis. Variable computations are inevitably required for conducting analysis. We were informed that this function was supported by Birhan Consultants. Local teams were relatively unaware of the formulas used for these computations, or additional data cleaning etc., that may have been conducted/required for the new computed variables. For example, the computed nutritional status variables usually require additioanal range/rule tests to delete outliers with Z scores outside the plausible range. It is not clear if such tests and associated data cleaning was conducted. · Data compilation and documentation This is a function that is usually delayed for after completing the data analysis and reporting. The extent to which the information is well documented, determines the usefulness of the data beyond the immediate analysis and reporting purposes. Others who may be interested in utilizing the data for programming or related analysis, as well as new staff joining SERA would require a well documented set of variable definitions. The quality of the analysis conducted with the data, and its reporting, is influenced by the thoroughness with which this is done. Data documentation is a time consuming task, and more so for large data sets such as SERA. It would include data files and associated variable listings and definitions from the questionnaires, their various modules, and all the computed variables. It is the view of the evaluators that additional attention will need to be given to this activity. Without this, future value, usefulness and application of the data collected will be limited. · Data Quality, and the tradeoff between quantity and quality There is inevitably a tradeoff between quantity and quality of any data collection exercise. This is why a well defined set of objectives, questions and analysis framework needs to be laid out prior to undertaking any data collection. The more focused the data collection activity is, the better the data quality is likely to be – given the resources available. In the case of the SERA data collection activity, there was a long list of indicators for vulnerability identified in the ‘National Guidelines’ and collected during the data collection phase of the project. Also, a variety of data sources were also to be used for these instruments: viz secondary data, household/women/child surveys, key informant intereviews and focus group interviews. The evaluators did not find a clear rationale for linking either the long list of indicators or the various data sources to the key questions that were posed for Vulnerability profies: viz who, where, when, why, what, how etc. 26 It is our assessment, that too much information may have been collected as result of lack of a clear analytical framework, and this is likely to have contributed to poorer quality of data than was possible, ceteris paribus. · Quality for what? Question of suitability for analysis, and relevance of data An evaluation of ‘data quality’ is very much linked to the way in which it is to be used. For example, qualitative data can be used for building narrative and ‘qualitative’ analysis. Such analysis is useful for understanding complex interrelationships that may not be possible in quantitative modeling types of analysis. It is generally recommended that qualitative analysis be used to understand the complexities, and once these are relatively well understood, that modeling be used to test specific hypotheses that might be relevant to explaining the relative importance of different causal factors. It is our assessment that the data quality, in terms of the information collected is well suited to documenting the local variety of local situations, and undertaking a qualitative analysis (ie. Descriptive analysis and narratives based on key informant and focus group interviews). It is unlikely to be suitable for quantitative modeling of causalities of vulnerability for two reasons: a) An analytical framework was prepared prior to the data collection, and as a result their may be missing variables required may not have been collected; b) data collection was not focussed to the necessary variables alone, and therefore data quality may have been compromised. For example, the ‘noise’ factor in key variables, eg., nutritional status of children, if it is large due to inadequate time given to accurate documentation of age of child. This is a common problem in large scale surveys – and can be discerned by ‘age clumping’ or a bulge of children at ages – 12, 24, 36, 48 months. This type of result can lead to a combination of bias in averages such as those used in descriptive analysis (a little less likely, since some mothers/families may average the age upwards and others may average it downward – though the downward bias is expected to occur more often) and more importantly, adds a big factor of ‘errors in measurement’ which can be detrimental to quantitative analysis, especially if the variable is used as a dependent variable. The evaluators found evidence of this kind of ‘age clumping’ for children in the anthropometric data of SERA. c) Level of awareness of data quality issues. There was great deal of effort made in data collection to ensure data quality – and this is very commendable. The hard work that went into this by SERA is very creditable. However, given the large volume of information collected, it is very natural to expect that the level of attention given to each and every element of the data was variable. The evaluators did not have enough time to go into each and every facet of data collected, but selected a few key aspects to assess this. Child nutritional status was one variable checked, since it seems there is an interest to use it in VP as a key indicator of vulnerability. Staff at the zonal and Regional level seemed to be unaware of problems of age assessment of children, and the impression was that if an effort was made to ask the mother – that was adequate to get an accurate age. While that is correct to a large extent, the fact that it is essential to get the month 27 and year of birth to accurately obtain an accurate age in months for the child does not seem to have been recognized. An accurate age in months is necessary to avoid the biases in nutritional status assessment. If the child is 15 months old, and the mother reports the age as 2 years – it gets converted into the data as 12 months. The result is an upward bias in the nutritional status – in terms of Z￾scores or % of median. When this occurs consistently, the result is the kind of age clumping seen in SERA data that was checked for this. In view of the above, the evaluators suggest that while the information is well suited to qualitative analysis of vulnerability factors and conditions, caution needs to be exercised in setting up modeling quantitative causal analysis with the SERA data. 4.2.4. Efficiency and Effectiveness The detailed indicators, methods of data collection and assessment instruments in the guideline were comprehensive and found useful not only to current efforts of the vulnerability profile development but other similar development endeavor in the regions. Primary data collection instruments were found to be very useful. In fact some of the hand tabulated HH and RRA indicators and instruments fore example have already started to assist in: · Conducting assessment of vulnerable groups and establish beneficiary database in the region. · Building the assessment and identification capacity, reviewing of existing food security related projects and guiding appropriate adjustment and interventions. The indicators selected are however, far more than required to measuring vulnerability. Collection of data for so many indicators, besides creating unnecessary burden in the data collection and management, makes it difficult to integrate information from so many sources. A clear link of each indicator and data source to the conceptual framework and key questions to be answered should be clearly identified, so that the data collection and analysis for vulnerability profiles meets its desired objectives. The Progress of Wereda vulnerability profile development has been slowed by a combination of huge data analysis demands, technical expertise of regional staff, and inability of Federal SERA to meet all the varying technical backstopping needs of the regions. For most of the 16 Vulnerability Profiles started in the Pilot Phase, only the Chapters 1-7 were written. These chapters are organized to present the results by each data source, and integration to address key questions (Chapter 8) and Conclusions (Chapter 9) are still remaining. These are planned to be finalized by the end of September 2001. We believe that lack of a clear and operationally simple analytical approach to guide data collection and analysis of the factors associated with vulnerability to disaster, is an important reason why the VPs are still incomplete. In the guideline extensive lists of indicators of vulnerability and different statistical tools for data analysis are provided but there is no a guide to analyze the vulnerability factors. In other words, there is insufficient foundation provided to connect the data and its 28 analysis to the overarching purpose and goal of disaster mitigation by understanding vulnerability and its sources. Even though Vulnerability Profiles have not been developed as planned, it is important to recognize that, given the dearth of any Wereda level information in the drought prone areas where SERA is working, this compilation of baseline information is still an invaluable contribution that has been made. It needs to be built upon, and not discontinued summarily. Even with simple descriptive tabulations, this information can provide a basis for designing interventions (albeit with additional made-to-order PRA or other data gathering to guide project formulation). However, there are no concrete examples of such use so far, since results have not yet been disseminated or shared informally with many program implementing groups. 4.2.3 Constraints and Recommendations The main constraints were: v National Guidelines overemphasized an extensive list of indicators and data collection from multiple sources but gave inadequate attention to the core concepts and analytical frameworks to be applied in their analysis v Overemphasis on data collection and over collection of data v Insufficient analytical basis for data analysis, so that the data could be efficiently used for addressing key requirements of VPs v Insufficient conceptualization on the content of VPs to meet their application needs and use by stakeholders v Data quality may not be suitable for complex multivariate models v Technical support and backstopping available not adequate to meet varying regional needs The main recommendations are: v Key requirements and content of Vulnerability Profile development should be identified clearly, ie. Which questions have to be addressed? How will this be done, ie., what is the analytical approach best suited to local needs and conditions? Who is the target audience also needs to be clarified – there is a danger that a one size fits all effort may not be of much use to anyone. v The data that has been collected may be the biggest asset and contribution of SERA. Its compilation and documentation to be usable for purposes of development planning and program design needs to be explored. v It is likely, however, that each has different types of requirements for data and types of analysis. To make VPs useful, the best approach may be a qualitative, narrative one, in which a combination of clear and interesting tabular information is interspersed with the rich KI and CGD material. This could be supplemented with detailed tabular appendices. v Vulnerability Profiles at wereda level are important to demonstrate the specifics of the local conditions and their disaster risks and vulnerabilities of different population groups. This should be an important bottom-up contribution to the top down and 29 secondary data oriented national and regional planning exercises. This work would add a rich contextual basis –and at the same time highlighting the variability in local conditions and needs for disaster mitigation – and ultimately for sound development. We recommend that this effort should therefore, be supported in future. v Future work on disaster mitigation should give greater attention to the needs for this type of work to bridge effectively the needs of both development planners and emergency response. This is a very tricky business from both a substantive and institutional perspective. Good disaster mitigation work is very important to incorporate into development planning. From a substantive perspective, disaster risk is a function of many factors in addition to income based poverty – and this will give development planners a broader context upon which to build their work. For example, sound environmental management, improving social and organizational structures may contribute to reducing disaster risk of vulnerable groups – in addition to improvements in income. From an institutional perspective, there is typically a disconnect between disaster response and development planning agencies. To the extent that disaster mitigation work is being developed within disaster response oriented institutions such as DPPC, the challenge to fully incorporate this work into development is huge. If the locus shifts entirely away towards development, there is a danger that the disaster-risk and vulnerability-to-disasters ways of thinking, and link with emergency response would get lost. This broader context can easily get sidelined in the more narrowly poverty or food security driven focus of development thinking, thereby ignoring key disaster risk elements such as sound environmental management, and building local institutions and capacity to obtain information, resources and respond to adverse ecological circumstances. This is an institutional challenge worldwide – and given the much greater level of appreciation for disaster risk reduction needs in Ethiopia – it is possible that such an interface could effectively be ‘engineered’ here. 4.3 Capacity Building 4.3.1 Various levels of capacity building undertaken There are several training and capacity building aspects that have been identified and carried out by this project: 1) For creating an awareness on the need, objectives and activities of the project, various orientation and motivation workshops were conducted at all levels starting from the federal to the wereda level. An orientation workshop to make the project operational in the four-target regions and the regions was organized, and they in turn, held workshop in the operational zones and weredas. The purpose of this workshop was to explain the objective of the project and to generate stakeholder interest in participating in the organization and management of the project. These workshops were instrumental in generating a generally high level of interest in, as well as expectation from the project. 2) Training of trainers on data collection was conducted at federal level. This was followed by regional and zonal level training’s. The purpose of this TOT was to 30 develop skills in data collection and HH survey/RRA at the federal (55 trainees), regional (11-15/region) and zone (72/zone) levels in - in total over 650 trainees. The training’s consisted of theoretical discussions on how to complete the questionnaire as well as practical interview of household head, eligible women anthropometric measurements of children 6 to 36 months old and measurements of MUAC of women in the reproductive age. In this survey, three groups of people were trained: the household enumerators, the supervisors of the household enumerators and the Rapid Rural Appraisal (RRA) team. Trainees were a combination of SERA staff, and VIMU and VWG members from Regions and Zones. 3) Training of field data collectors at Wareda level on household surveys, Key informant interviews, FGDs, women and children’s nutritional surveys, and secondary information availability. The personnel involved were from different sectors, and other Wareda level workers that were hired as data collectors on a short term basis by SERA. 4) Training of computer data entry and processing techniques at Zonal level (primarily SERA staff) 5) Capacity building for Vulnerability Profile related data analysis and report preparation to Zonal and Regional level VIMU staff (primarily SERA staff) A series of training activities were designed to improve data management, integration, and analysis capacity of the regions. The data management and analysis training workshops was conducted by the SERA Project, in collaboration with Birhan Research and Development Consultant. In this training 9 participants from each of the four regions representing mainly the DPPB and regional SERA project offices have participated as per the following : - developing skill in primary data management and analysis by training 6 federal and 14 regional staff ; - Developing skills on data integration and summary for VP development by training 14 regional and zonal staff for 4 days. In addition to the basic computer training, GIS and SPSS training programs were also conducted for federal, regional and zonal focal office staff. In general the efforts made for capacity building has assisted regional project offices to undertake the project activities and to collaborate with other line departments. 4.3.2 Efficiency and Effectiveness In terms of effectiveness of the training and capacity building efforts, there was diverse feedback received from federal, regional, zonal and wereda staff who were interviewed. Regarding the various capacity building components. Regarding the relatively basic training’s given at the regional level, many interviewee opinions during the evaluation mission, attest to an increased and improved skill in data collection, assessment and survey methods at all levels. The wereda and zonal officials in particular, reported that the training has helped them with basic survey skills that are useful in the conduct of their jobs. They also valued the basic computer training, with data entry and management 31 skills training. Feedback was mixed for the higher tier of skills training provided – especially on analytical methodologies related to VP development. Opinions provided by regional, and zonal SERA staff, and VWG members however, revealed that the training’s given at the federal level on higher tier skills tended to be too theoretical and its design didn’t take in to account the capacity and actual needs of the respective regions.4 Turnover also tended to be higher for the more skilled staff. The evaluation has reviewed the background documents and content of the major national level workshops that were oriented to Vulnerability analysis – such as the August 1999 workshop on ‘National Guidelines’ and in March 2001 on ‘Vulnerability data management and analysis’. These efforts are commendable in their broad approach to imparting a variety of key concepts and methodologies. This kind of knowledge is necessary for building an understanding of the complexities involved. This is clearly a valuable contribution to human capacity building. In attempting to understand the responses to this effort from the Regional SERA teams’ perspective, it appears that there may not have been adequate consultation between federal and regional teams as to the specific requirements as a follow up to federal workshops. However, it is difficult for us to fully understand actual events that may have contributed to such a discontinuity between federal SERA and regional SERA expectations and satisfaction with the level of training provided. It is likely that not while there was great emphasis on providing a broad based understanding, not enough attention may have been given early on to setting up the specific analytical frameworks. Both the Tigray and Amhara regional teams expressed their dissatisfaction at this, and their frustration at not having sufficient technical input for this at any time. The project’s institutional strengthening mandate has been interpreted as working primarily with government - the DPPC system in particular. This approach, open and participatory in nature, and closely integrated with government goals and objectives, has earned SERA staff the trust and respect of its host institutions. It may also have helped DPPC and other regional hosts of SERA project to incorporate elements of disaster prevention into their work. SERA input into DPPC plan development, an essential element of institutional strengthening in the broadest sense, has been promoted. The DPPC/F, DPPC-Amhara, and FSD-Tigray have all seconded their own staff to the project, which it a testament to its perceived value, and also a promising step for future capacity building. Given the level of confidence and the close working relationship between SERA and DPPC, the Evaluation Team sees the opportunity for significant future input in the planning and research areas. 4 Some typical observations were: “Though there has been much technical support from the Federal DPPC, the support particularly in the area of data analysis and integration has been inadequate” and “These training’s given both at regional and federal level has been useful but, there design didn’t take our capacity and actual needs in to account.” 32 4.3.3 Constraints and Recommendations The main constraints were: v Substantial delays in organizing key capacity building oriented workshops, such as the data entry and data analysis workshops. v Inadequate attention to level of analytical capacity in regional teams v Insufficient flexibility to address varying regional needs v Inability to retain trained workforce The main recommendations are: v Extend duration of methodological training workshops, and make them more participatory, need oriented and demand driven. v An ongoing process in which the learning is not only of key concepts and principles but oriented to tasks at hand in an ongoing way is highly desirable. v Establish a foundation level of core team capabilities required for basic VP development v Tailor recruitment and training activities to enable attaining the level of core capabilities v Expand capacity building of permanent govt. staff and their involvement in SERA v A sharing of inter-regional experiences should also be considered as part of capacity building 4.4 Research on Root Causes of Vulnerability 4.4.1 Background on identification of SERA’s research component One of the SERA project objectives is to undertake in-depth vulnerability research in identifying root causes of vulnerability and find out ways to tackle them as prevention and preparedness policy. In this regard, the following 17 research areas categorized into five programs were identified in the Ghion Workshop in 1997 have served as a basis for selecting the research topics to be undertaken under this project. (see the 1997 Ghion workshop proceeding). These are: · Evaluation vulnerability profile policies, (3 topics, 1 is considered) · Impact evaluation of the program/project, (3 topics) · Analysis of the root cause of vulnerability, (4 topics, 4 are considered) · Population response to vulnerability, (3 topics) and · Institutional and human resource capacity buildings (4 topics) Out of these 17 research areas the following five research topics were selected as priorities to be financed by USAID in the 1997 Project document. 1. Analysis of drought forecasting capability in Ethiopia 2. Root causes of ecological degradation 3. Indicators of vulnerability of, epidemics and copping strategies 33 4. Population Pressure and Carrying Capacity of land and off-farm employment 5. Targeting indicators of the most chronically vulnerable groups 4.4.2 Definition of goals and concept of component In the 1997 project document identified this component as one that would be collaboratively undertaken by DPPC and USAID, and would be oriented to identifying “more efficient and effective approaches to using food resources to target vulnerable populations…” At this point it was clearly linked to use of food aid and to identify ways of using it to promote development through relief measures. In the 1998 Revised Project document, this activity was slightly reformulated in its goals and concept. It is possible that this reformulation was based on the feedback received from regions. In this document, the Research component is seen as a way to link Vulnerability Profiles to Response Packages. One of the key objectives was stated as follows: “strengthen response mechanisms and development interventions through incorporation of the results of vulnerability profiles and research results”. This perception is also reflected in the National Guidelines, and in the understanding of regional SERA staff interviewed. It is however, not clear at any point, how this connection between the research outputs and VPs would be made to develop the Response Packages. 4.4.3 The approach – a federal orientation Though the concept and goals of this component shifted between the 1997 and 1998 documents, the original federal orientation of the work remained. In the later document it was stated that “vulnerability research will be coordinated at the federal level by SERA staff in close coordination with PPPD (office of DPPC.) It will involve different government research institutions, or non-govt. organizations, and the Regions in actual research”. Regions have been consulted with regard to selection and prioritization of these research topics from the 17 research areas identified in the 1997 workshop. Accordingly regions have been able to reflect their prioritized research topics that need to be addressed as a priority under the project through letters written to the DPPC. The evaluation team was informed that the regional expressions of priority were incorporated into the final choices made on research topics. Federal DPPC/SERA have developed 4 TORs for: - Vulnerability to major types of flood in Ethiopia: prevention and preparedness - Interrelationships between population pressure, carrying capacity of land and off/non￾farm employment - Vulnerability to epidemics in the context of drought, disaster prevention and preparedness - Analysis of drought forecasting capabilities in Ethiopia 34 In addition, a fifth topic – “Targeting indicators of the most chronically vulnerable groups” was covered through a buy-in of ongoing work being done in Ethiopia by an international NGO. 4.4.4 Progress made While three of the research topics for which TORs were developed, are among the above 5 prioritized areas, a new inclusion (Flood Vulnerability in Ethiopia) was not foreseen in the initial project agreement. Of the four TORs, only for two research topics (i.e. NMSA and ENHRI) have the technical proposals have been reviewed and accepted by federal agencies, and only one started implementation (Indicators of vulnerability, epidemics and copping strategies). The implementation of this component has been slow, and there is considerable uncertainty of its relation to VPs and responsiveness to regional and wereda level response packages development. ? Root causes of ecological degradation The objective of this research topic was to assess, analyze and prioritize the root causes of ecological degradation and their role in vulnerability to different disasters. Though this topic has been identified by all regions as the most high priority area for research study, particularly from the standpoint of developing response mechanisms for reducing vulnerability to disasters, and was also indicated as priority research area in project document, SERA has not included this work so far. It is the opinion of the Evaluation Team that ecological factors are key contributors to vulnerability to disasters. Understanding them, and responding to them, however requires a localized approach that can not be generalized beyond many local contexts. For this reason, this component in particular, needs to be included in a solid way at the wereda level. An option that needs to be considered is using the rich qualitative information from KI and CGD in the SERA effort to develop this research topic. Doing this in a localized way will also help development oriented NGO programs to incorporate this key element in their efforts. ? Drought Forecasting Capability in Ethiopia The general objective of the study on” Drought Forecasting Capability in Ethiopia” is to assess and identify the root causes of vulnerability to drought in the socio-economic system of the country and develop preventive and preparedness strategies and measures. TOR for this research topic was prepared and after reviewing the technical proposal submitted from NMSA, implementation agreement was signed between the DPPC and NMSA in May 2000. However, for some of the following administrative problems and constraints the study has not yet been started. Some of the problems associated with this are: 35 · Delays in the approval of payment arrangement and permission from the Civil Service Commission, which took about 8 months after the request. · Uncertainty of the Lead Agency (NMSA) of its mandate and capacity to carry out the research on vulnerability to flood, and required top management involvement. Attention is needed on how this research topic would contribute to responses to mitigate disaster risk from drought. From a Federal level perspective, it would provide a type of ‘early warning’ of impending problems. However localized responses may be most appropriate, eg., in expanding seed banks and other protective local mechanisms. It was not clear to the Evaluation Team, in the limited time available, to what extent this function is already being covered by other ongoing activities, eg., FEWS, and therefore not required. This should be examined. If it is found that this activity would simply be duplication of existing work, it should not be moved to implementation. Instead, ways of linking existing ‘early warning’ type efforts to local response mechanisms could be considered. ? Vulnerability to Major Types of Flood in Ethiopia, The objective of this study is to assess types of flooding and identify major flood prone areas in the country and propose prevention and preparedness strategies in short and long run. The study was envisaged to be implemented by Ministry of Water Resources. No action has been taken on this by them so far. Floods tend to be localized events in most years, and are often exacerbated by the same ecological degradation factors such as deforestation and soil erosion, that also exacerbate droughts. However, communities affected by these events are often different than those impacted regularly by drought. Ecological degradation of the types mentioned above, not only trigger floods and increase the intensity of droughts, but also hinder development efforts such as providing irrigation facililties because of excessive silting problems. It is important for these connections to be made in work undertaken on flood disasters. It may be easy to overlook their importance since these are often localized problems, that may appear to be dwarfed by the larger drought issues. The connections between these, and the opportunity to link disparate communities to engage in mutually supportive activities is a newly emerging one showing great promise, internationally. This is an idea that could be explored in developing this research theme in Ethiopia. ? Population Pressure and Carrying Capacity The general objective of this study was focused on studying the interrelationship between rural population pressure, carrying capacity of the land and off/non-farm employment 36 opportunities in the selected weredas exposed to high vulnerability to food security. This study topic, was envisaged to be undertaken by Addis Ababa University (IDR/DTRC), Though TOR for both of the above two-research topics was prepared by the SERA project, technical proposal has not been received from those agencies expected to participate in the study. One of the major problems with regard to this is that since government institutions are preoccupied with their regular duties they could not give priority for such tasks, as it is secondary duty/responsibility to them. Population pressure and carrying capacity is also an issue that is closely related to the ecological conditions, as well as to other conditions such as demographics, off farm opportunities, level of investment in agriculture and rural infrastructure. As a distinct research topic, it may however, be hard to undertake in all its complexities. Unless this is redefined, it is not clear if a useful research could emerge from this topic, both in and of itself, and in a way to provide a contribution to response measures. ? Targeting indicators of the most chronically vulnerable groups This study was dropped by the DPPC taking into account the National Food Aid Targeting Guidelines issued earlier by the DPPC developed would meet the objective of the study. This guideline was developed on the basis of previous targeting study entitled Targeting of Food Aid in Ethiopia by the SCF-UK and consists of procedure to be undertaken in the process of targeting relief food aid for disaster affected population. ? Vulnerability to Epidemics: Prevention and Preparedness The general objective of this research topic was to study the main causes and risk factors of vulnerability to major epidemics in drought prone and other areas, identify its health consequences, and recommend feasible prevention and preparedness models and strategies. An agreement for the implementation of this study was signed between DPPC and EHNRI in August 2000. Due to some administrative and financial arrangement for the project the project started operation only in January 2001. The study is being implemented by EHNRI and expected to be completed in the end of the year 2001. This is the only element of the “root causes” research component of SERA that has been effectively commissioned so far. The implementation Status of Vulnerability to Epidemics is given in Annex 3. 4.3.2 Constraints and Recommendations The main constraints were: v Though the research component is expected to tie in with the VPs to help in developing the Response Packages, it is not clear in the design of these studies precisely how this connection would be made v Little progress in implementation of this component may have jeopardized any conceptualization of Response Packages by regional teams 37 v Lack of regional consultation and input in TOR development process, or in the implementation of research (eg. Vulnerability to epidemics work did not consult with regional teams, or consider overlapping its sites with SERA sites for VP development). The main recommendations are: v Rethink this component to identify its key goals and required outputs v Given the delays in starting this work, an abridged program of work should be envisaged, one that is clearly linked to the needs of the VPs and developing useful Response Packages 4.5 Response Packages 4.5.1 Concept and Goals In general terms, the idea of Response Packages emanated from the 1997 Ghion Workshop. It was generally agreed by all the stakeholders at this Workshop, that better understanding of the root causes of the vulnerability to disaster was required to enable appropriate response interventions at different levels. In the original project document, the VP and research were seen as a connected set of activities that would contribute to Response Packages. The RPs were envisaged as “regional and zonal contingency plans that detail appropriate interventions for particular disasters, populations and localities”, and would be based on VPs. Thus, in the original formulation, these appear to have been thought of as a set of local projects on the shelf-5 that could be utilized by NGOs or by the Govt. in case of a disaster. Availability of such potential project ideas that are suited to local needs are one way to enable disaster response measures to better address longer term development needs. The thinking about Response Packages had changed in the 1998 Revised project document. Here, two types of RPs are envisaged – one that would combine results of VPs and vulnerability research to develop RPs, and the second type of “bottom-up planning of response packages in some Weredas and zones”. In the 1998 Revised Project Document, the VP and Research components were separated, and the Response Package idea, though still included became somewhat unclear. It was envisaged that these two separate components – one carried out at local wereda level and the other at the Federal level, would somehow be combined to develop responses (or response packages). How this would actually take place, was not clear. This issue was also not addressed in the National Guidelines for VPs, or since then in any of the documents reviewed by the Evaluation Team. There is no explicit basis for the assumption that once all the VPs and Research is done, that RPs would emerge, or would be facilitated. This assumption, however, appears to have been accepted, despite little attention given to interfacing VPs and Research, or 5 This concept resembles the Employment Guarantee Schemes in drought prone areas, such as in Maharashtra, India. When there is a disaster event, or risk of one, these project ideas are put into action. 38 explicitly considering how the two would be combined, or who would develop Response Packages. For example, the site selection for the Vulnerability to Epidemics work now being carried out did not consider need for any interface with VP work and weredas where it is being carried out. We are left with the perception that it is expected that ‘others’, eg., NGOs, planning depts, or sectoral units would somehow be able to utilize the information, once it is all available. This may not be a very sound proposition. While information/research is useful, especially when it has a practical and applied focus, unless the response/application of the information is explicitly incorporated in the formulation of the activity, it is not likely to be successful. In the case of the SERA project, the response function is explicitly part of the project deliverable, unlike other research, and therefore this should have been considered explicitly. It is worth mentioning that an additional, and interesting approach to RP formulation was introduced in the 1998 Revised Project Document. Although it was introduced as a pilot effort, to be tried out only in a few weredas, it opened an opportunity to develop RPs with local community participation. This option also does not seem to have been picked up in SERA implementation – which given its full attention to the data collection and processing effort. Given the involvement of local leaders and communities in the data gathering which included discussions on disaster risk, vulnerability and causes – there was (and still is) a good opportunity to build on these exchanges. This could best be accomplished in cooperation with local NGOs, utilizing information that has been collected, and re-engaging with the communities to create investment and development options. Working in this way with local NGOs and communities could offer a good outreach opportunity for achieving local level benefits from the SERA effort. In the interviews conducted by the evaluation team at regional, zonal and wereda levels, it was very evident that the promise of response packages was a major factor in the enthusiasm with which stakeholder participation at all levels in the project had been forthcoming. Therefore, it is the view of the Evaluation Team, that during the completion of the pilot phase of SERA, this component be given priority. 4.5.2 Progress made So far, there is nothing undertaken with regard to the development of the response packages. As explained above, this is partly a result of the way in which the Revised Project Document of 1998 was formulated which states that completed VPs and indepth research would contribute to formulating RPs. Since neither VPs nor the research component has produced its outputs so far, the RPs are also not available. Independently of SERA project outputs of VPs or indepth research, to the extent that the data collected may have been used by NGOs working in these weredas as background information for their program development, there may already have been an application of the work that has been done by SERA. This is based on information received by the Evaluation Team, but no independent verification of this was possible. Therefore, no 39 specific suggestions on the relative usefulness, or how it could be improved can be made in this report. In Tigray, some tools developed by SERA have been applied to develop Response Packages in areas (other than SERA pilot weredas) for food security improvement. It is suggested that the experience from the Tigray food security effort be disseminated to all SERA teams to inform them on how information can be used for local level program/project planning. Given the limited attention to this deliverable in SERA project implementation to date, it would be useful to consider concrete information sharing between regional teams and federal SERA on their outreach experiences and approaches that could be fostered in completing the Pilot Phase of the project. 4.5.3 Constraints and Recommendations The main constraints were: v There is no uniform or agreed operational guide on how and who would be responsible for developing such RPs. v Unavailability of Vulnerability Profiles or Research outcomes that can provide the a basis for response packages, planning or program support for targeting vulnerable populations and identifying interventions at the different levels. v Insufficient VWG or other stakeholder involvement and consultation after the data collection phase for identifying possible applications by different groups of the information and analysis. The main recommendations are: v This component should receive high priority attention for the completion of the pilot phase. It is unlikely that the RP concept of the combination of VP and indepth research as contributing to RPs will materialize in the foreseeable future. Therefore alternative approaches should be considered, including implementing the “bottom up” planning response approach in cooperation with implementing agencies. v Given the high level of expectations that have been generated for developing response packages, especially at the wereda and zonal levels, it is important that a clearly visible effort should be made to generate some outputs in this regard. A great deal of involvement and input from local leaders and officials has contributed to SERA, and this should not become one more example of broken promises made by development agencies who seek cooperation by local communities. v Stakeholder consultations using selected VP information could be used to facilitate wereda level bottom-up response packages. 4.6 Overall Efficiency One of the strengths of the SERA intervention is the participatory and cooperative approach adopted, which is certainly appropriate for capacity building and is praised at all levels of government. SERA has also demonstrated efficiency in fairly good quality data collection, data entry and cleaning. The data was processed using SPSS for widow’s 40 software and other compatible packages. But the lack of integration of activities within and outside of the project’s control has resulted in modest to low efficiency in the use of this equipment. This is a complex project in many ways. First, it has aimed at creating capacity to transition the formerly relief oriented DPPC towards disaster mitigation, and therefore to bridge the disaster-development divide. Second it undertook an ambitious and comprehensive data collection task at the wereda level – in many cases where little of no information was formerly available. Third, the concept of disaster risk and vulnerability is a complex, multifaceted one, and the project aimed to get a broad based consensus on the meaning and approaches to analysing it. And fourthly, project began with a well intentioned attempt to decentralize to regions the main task – the development of VPs – something that was not envisaged in the original project document. A lot of progress has been made in the four years since the project was initially launched. This includes comprehensive regional level discussions with stakeholders to engage their participation, and which subsequently produced the Revised Project Document of 1998. Further to that was setting us the implementing arrangements at federal, regional, zonal and wereda levels - that included staff not only hired by SERA but also staff of host institutions and input in various capacities by numerous stakeholder organizations. A large and rich data collection effort in 16 pilot weredas has been carried out. A large number of trainings have been carried out for staff at all levels, and data analysis and write up of VPs is underway. There are a number of factors that have contributed to the low observed level of output to date, as compared with what had been anticipated or planned at the outset. These include a combination of management and technical issues that have been discussed in the sections relating to the five main outputs – Institutionalization of disaster mitigation work in DPPC structure, Vulnerability Profiles development, capacity building, indepth research, and Response Packages. 4.7 Overall Effectiveness The SERA VP development component has been confronted with start-up difficulties and delays that have had an impact on the timely achievement of results. In general, the activities planned by SERA for this component are progressing, but are significantly behind schedule. Given the methodological difficulties encountered in making aggregation and comparisons among the collected data and completing vulnerability analysis or research on root causes of vulnerability, it is uncertain if concrete results leading to recommendations and response packages can be completed by September 2001. As explained above, it is the opinion of the Evaluation Team that without having a clear analytical link with the goals and conceptual basis the VP process has been stymied. However the information that has been collected is a valuable resource, and based on the work done so far, an effort is needed on how to best present it to users so as to achieve a 41 favorable outcome and impact. The National Guidelines have been fully utilized for Vulnerability Profile development by Federal and Regional SERA teams for implementing this plan of work. This has involved development and standardized use of the 5 main data collection instruments in all 16 pilot weredas, an efficient data collection system for a total of about 9,900 households. Expected results on Vulnerability Profiles, Research on Root Causes of Vulnerability, and Response Packages have not been achieved. The overall perception over time of the importance of bridging emergency response and development by developing effective disaster mitigation effort has grown over time, creating a greater demand for SERA results for mainstream planning and food security efforts. Lack of visible results has generated some impatience with its progress. This is understandable, given the urgency and need to respond to the chronic and acute vulnerability faced by large populations. 5. OVERALL RECOMMENDATIONS 5.1 Continuing Rationale 5.1.1 DPPC and USAID Priorities The SERA project represented the important priority for DPPC to build capacity for addressing long term development needs as part of its relief efforts, so as to reduce vulnerability to disasters. This shift in emphasis requires a gradual accumulation of expertise and information, as well as an ability to engage substantively with various sectoral activities. Similarly, USAID-Ethiopia has placed disaster mitigation as one of its strategic objectives for the new 5-year project cycle. It is clear that since the origin of SERA, the importance of its goals and objectives are perhaps even stronger now than ever. 5.1.2 Delays in the face of high expectations A major inducement for the participation of weredas, zones and regions in the largely research oriented SERA project, was the promise of results and information that would provide the basis for making sustainable improvements in their vulnerability to recurring disasters. It is important that the project be able to fulfill on this promise. 5.1.3 Other Initiatives that have increased relevance of SERA outputs Many developments in Ethiopia have increased the relevance of SERA. Among these are: - the progress of decentralization, and increased need for local information and expertise to address regional priorities; - the growing institutional support for food security as a central element of development programs. This increases the need for SERA information and analysis; - the strengthening of regional planning departments, offers potential partners for SERA to be able to utilize a multisectoral forum for the outreach and application of results. 42 5.1.4 Other Initiatives that could replace/interface with SERA At the same time, the rapid pace of new initiatives could also easily overtake SERA, and make it a redundant exercise. Efforts are underway, eg. In Tigray and Amhara to strengthen their regional planning data bases – by radically improving secondary data quality and reporting mechanisms, including using GIS techniques. Also, donors such as the World Bank, EC and others supporting the food security strategy work in regions, are considering developing a wereda level information base. These developments make it important for SERA efforts to be an efficient and streamlined one, that can deliver useful products to its stakeholders. The potential for being able to interface with these new efforts can improve the reach and usefulness of SERA investments. 5.2 Urgency of Producing Quality Outputs for Pilot Phase 5.2.1 Vulnerability Profiles There are several sub-outputs of the Vulnerability Profile development – national guidelines, useful baseline information for Weredas, and outputs that allow resource allocation at different levels. Wrap up of the present scope of work should focus on: a) producing a usable information base that includes data compilation and documentation; b) b) Vulnerability Profiles that are provide a clear picture of the baseline situation in the weredas. Given the absence of clear analytical frameworks, it is considered feasible that any sound causal modeling type of analysis will be possible with the data available. Therefore fuller use should be made of clearly presented descriptive tables and narratives based on the Key Informant Interviews and Community Group Discussions; c) c) Statistical tables that document baseline conditions, by, for example agro ecological zone and/or other sub wereda classifiers – which could be used for program planning and monitoring conditions in the future. 5.2.2 Response Packages Given the lack of progress so far in VP and Root Causes Research, as well lack of clarity on how and if they would contribute to the envisaged Response Packages, priority should be given to “bottom up response packages” in the wrap up to this phase of the work. A priority at Wareda level, would bring some results to the communities and their leaders who have been supporting this work. Other options involving stakeholder consultations at different levels could also be explored. In any case, more attention is needed for this component. 43 5.3 Follow up to present SERA Project for 2001-2003 5.3.1 Redefining Federal role Redefining the federal role in SERA will be useful in light of the need for decentralization and regional ownership for generating value and relevance. By focussing on strengthened technical support, ability to respond to varying regional needs in providing inputs, including capacity building, a better working relationship can be established that would produce superior results. Oversight and monitoring progress should also be part of the federal responsibilities. 5.3.2 Expanding Regional responsibility In particular, strengthening the ability of regions to manage the SERA project will be useful. Regional budgets and workplans that can be monitored would help in smoothening the administration, reduce additional layers of management, and also increase a sense of responsibility in the regions for producing results. This increasingly is the model being followed by projects in Ethiopia in which regional level need based responses are required, eg. The food security strategy work currently being developed by EU. If needed, attention can also be given to strengthening institutional capacity in regions to administer and report on finances and also to implement procurement and recruitment needs. Such an approach could also be tied to a higher level of explicit institutional commitment for providing a part of the staffing needs. Given the widespread acceptance of the disaster mitigation focus by now, the likelihood of its acceptance is also high. Such institutional commitment should also be balanced with the need/ability to effectively make a substantive link with a combination of disaster response/early warning efforts on the one hand, and with development oriented efforts such as food security on the other. Better integration would also help with the staff turnover issues that have been faced by the regional SERA teams, and dissatisfaction with the temporary terms of employment currently available under DPPC regulations. 5.3.3 Establishing a more focused concept, methodology and data needs In the follow up to the present SERA project, there is a need to revisit the National Guidelines to establish a more focused concept and analytical methodology for Vulnerability Profile development so as to more effectively utilize the available data to develop vulnerability profiles. Additional data collection in expansion phase weredas should not be started until the pilot phase activities are satisfactorily completed, and the lessons learnt can be integrated into future work. 5.3.4 Addressing the issue of Institutional Sustainability There is a variable level of SERA integration into host institutions in different Regions. Though DPPC/Bs in Regions are primarily the SERA impelementers, in some cases other departments, eg., the Food Security Desk in Tigray, and Planning department at zonal level in SNNPR are involved. Such variation is certainly appropriate given the local situations. However, irrespective of the precise location, the issue of sustainability and 44 the commitment to integrate day to day functions into departmental responsibilities is necessary if there is to be institutional commitment to enlarging disaster mitigation work in DPPC or other Govt. agencies. The location of this work and its formulation also has to consider the tricky but important link with both early warning and disaster response elements managed by DPPC and mainstream development efforts such as food security oriented strategies and schemes. 45 PROJECT RELATED DOCUMENTS REVIEWED BACKGROUND REPORTS 1. Workshop on “Vulnerability in Ethiopia: From Disaster to Development” Papers presented in the Seminars and Final Report, June 1997. Addis Ababa 2. Transitional Govt. of Ethiopia (TGE), Directives for Disaster Prevention and Management, October 993 3. Transitional Govt. of Ethiopia (TGE), National Policy on Disaster Prevention and Management, October 1993 4. USAID-Ethiopia, Integrated Strategic Plan, FY 2001-2006 5. National Food Aid Guidelines, DPPC, Addis Ababa, November 2000 6. Poverty Situation in Ethiopia. Welfare Monitoring Unit, MEDAC, March 1999, Addis Ababa 7. Vulnerability Mapping and Geographical Targeting: An exploratory methodology applied to Ethiopia, Patrick Webb et al, IFPRI, Jan 1994, Washington DC SERA DOCUMENTS 1. National Guidelines for Vulnerability Profile Development, DPPC/SERA, 2nd Draft, Sep. 16, 1999 2. National Guidelines for Vulnerability Profile Development, DPPC/SERA, Secondary Data Tabulation Sheets, Revised, August 24, 2000 3. Proceedings of Workshop on National Guidelines for VP Development, DPPC/SERA, Addis Ababa, August 23, 1999 4. Guiding Principles for Coordination and Standardization for Vulnerability Profile chapter analysis and write up. DPPC/SERA Project, Revised, July 23, 2001 5. Technical Review on Draft National Guidelines on Vulnerability Profile Development of the SERA Project. By VAG Meeting at DPPC, May 27, 1999 6. Federal VAG – Comments on the National Guidelines for VP Development by WFP (John McHarris), EU Food Security Unit (Mihret Bizneh), MEDAC (Debebe Habtewold), Ministry of Agriculture (Mathewos Hunde), Save the Children, UK (Cassandra Chapman) 7. TOT Manual for Household Survey, SERA/DPPC 8. TOT Manual for Selected RRA and Qualitative Methods, SERA/DPPC, August, 1999 9. Vulnerability Profile Development. Papers presented at SERA Data Analysis Training, DPPC, March 26-31, 2001 10. Workshop Report from the Data Analysis training of March 2001, Birhan Consultancy, June 2001. Addis Ababa 11. Terms of Reference for Indepth Research, DPPC/SERA a) Vulnerability to major types of flood in Ethiopia – Prevention and Preparedness (Revised Draft) June 6, 2000 b) Interrelationshipa between population pressure, carrying capacity of land and off farm employment (Revised) November 2000 46 c) Analysis of Drought forecasting capacity in Ethiopia. February 2000 d) Vulnerability to Epidemics in the context of Drought, Disaster prevention and preparedness. Revised draft. Feb 2000 12. Women’s Survey Questionnaire, SNNPR/SERA Project 13. Household Survey Questionnaire, SNNPR/SERA Project 14. Proposal for Expansion and Extension of SERA Project, SERA/DPPC, Dec. 2000. 15. A Draft Proposal for the Training of Primary Data Collection Instruments and Procedures, SERA/DPPC, Dec 1999 16. Anthropometry – Short Note for SERA Training, January 2000, Awassa (Amharic version) 17. Guidelines for Data Collectors for Vulnerability Assessment Sample Survey, Dec 2000 (Amharic version) 18. Coded value labels and Defined variables, Household Questionnaire, SNNPR 19. Results Report to USAID for USFY 2000, DPPC/SERA Project, (Oct 1999-Sep 2000) 20. Annual Report to USAID, Important Achievements and Problems/Concerns, DPPC/SERA for USFY 2000 21. A Performance update for USAID/FHA, SERA, Dec 1998 22. Six Month Progress Report for SERA Project, Mar-Aug 1998 23. SERA Project Monthly Reports, March 1998 to July 2001 24. Draft Vulnerability Profiles available to Evaluation Team a) SNNPR, Hadiya Zone, Badowacho Wereda (Chapters 1-7) b) SNNPR, Wolayta Zone, Humbo Wereda (Chapters 1-9) c) Oromiya, E. Hararge, Gurawa Wereda (Chapters, 1,4,5,7,8) d) SNNPR, Lemu Wereda, Chapter 7 e) Amhara, Tach Gayint Wereda, f) Tigray, Saese Wereda (Chapters 1-5, 8) g) Tigray, Pheferom Wereda (Chapters 1-7) h) Oromiya, Dera Wereda (Chapter 1-7) 25. Tables produced from Household Survey Data, Sasie Wereda, June 2001 26. Tables produced from Household Survey Data, SNNRP, SERA Project, June 2001 TECHNICAL DOCUMENTS 1. Development in Disaster Prone Places: Studies in Vulnerability by James Lewis. Intermediate Technology Publications, 1999. 2. Famine Early Warning and Response – The Missing Link by Margaret Buchanan-Smith and Susanna Davies. Intermediate Technology Publications, 1995. 3. ‘Getting the Scale Right: A comparison of analytical methods for Vulnerability Assessment and Household-level targeting’ by Linda Stephen and Thomas Downing. Disasters Vol 25 (2) 113-135, 2001. 4. ‘Targeting the Vulnerable: A review of the necessity and feasibility of targeting vulnerable households’ by Susanne Jaspers and Jeremy Shoham. Disasters Vol 23(4) 359-372, 1999. 47 5. Identifying the Food Insecure: The application of mixed method approaches in India by Kimberly Chung, Lawrence Haddad, J. Ramakrishna and Frank Riely. IFPRI 1997. 6. Living with Drought: Drought mitigation for sustainable livelihoods by Astrid von Kotze. Intermediate Technology Publications, 1999. 7. Rural Vulnerability to Famine in Ethiopia by Mesfin Wolde-Mariam. Intermediate Technololgy Publications, 1986. 48 PERSONS MET ANNEX 1 Project Documents (1997, 1998) ANNEX 2 Field Trip Report of Evaluation Mission 49 ANNEX 3 Root Causes of Vulnerability – Implementation Status of Vulnerability to Epidemics component ? Vulnerability to Epidemics: Prevention and Preparedness The general objective of this research topic was to study the main causes and risk factors of vulnerability to major epidemics in drought-prone areas, identify its health consequences, and recommend feasible prevention and preparedness models and strategies. An agreement for the implementation of this study was signed between DPPC and EHNRI in August 2000, however, due to some administrative and financial arrangement for the project the project started operation only in January 2001. The study is being implemented by EHNRI and expected to be completed in the end of the year 2001, i.e. at least one year behind. The implementation Status of Vulnerability to Epidemics follows. 3. Implementation Status of Vulnerability to Epidemics: The general objective is to study the main causes and risk factors of high vulnerability to major epidemics in drought-prone areas and its health consequences, and to recommend feasible prevention and preparedness models and strategies. An agreement for the implementation of this study was signed in August 2000, and however due to some administrative and financial arrangement for the study was started in January 2001. The implementation of this study is underway by EHNRI, which would be complete in by the end of this year 2001. The project cost Birr 954,000 and the fund released in February 2001.The major objective in this study includes: Specific objectives 1. To assess the occurrence and incidence of the most common and devastating types of epidemic and communicable diseases before, during the after periods of significant drought. 2. To analyze the causes and risk factors in vulnerability to epidemics in drought-prone areas, including their geographic distribution and high-risk population groups. 3. To assess the public health impact of these epidemic diseases on health status (including morality/morbidity, disability and malnutrition) in drought-related epidemics. 50 4. To assess the existing capacity of public health and disease surveillance systems in drought-prone areas, and recommend the most effective and feasible preventive health and nutrition measures to reduce vulnerability to drought-related epidemics. 5. To assess the current local response capacity and recommend a model and strategies for effective preparedness, response and rehabilitation mechanisms when drought￾related epidemic occurs. Methods employed to the study 1. Collection of secondary and available data for epidemiological analysis 2. Collection of retrospective and cross-sectional primary data in drought-prone areas 3. Undertake case studies in different agro-ecological zones and in current drought￾affected areas, including laboratory and anthropometric components 4. Dissemination of findings at national and regional levels 5. Recommendations for interventions The study has completed the 1st round data collection in 20 woredas all over the country and two-quarter progress reports have been submitted. The 1 st round data collection was arranged in four directions (Northeast, Northwest, Southwest and Southeast) by four groups to achieve the following four major activities: 1. Secondary data collection from 53 sites 2. Retrospective data collection from 18 weredas 3. General case studies in 6 weredas 4. Specific case studies in 6 weredas In general the planned activities in the 1 st round field trip in the four directions were successfully completed except for secondary data collection which still needs some follow up to get the necessary information from distributed sites. Accordingly the following achievements have been performed: a) Secondary data information has been collected from 14 woredas, 12 zones and 2 regions. b) Retrospective data has been collected through conducting: · 85 Focus group discussion · Key informants interview for 728 individuals · House Hold Data collected from 7200 sample HH · Nutritional qualitative information collected from 720 samples c) General case study: Different samples have been taken from over 3000 individuals in six weredas d) Specific case studies: the following specific case studies have been undertaken · Samples have been taken from patient for different diseases · Anthropmetric measurements 51 · Samples from mothers and children’s for hemoglobin · Water samples · Information from traditional healers and clients · Medical plants collected · Saliva samples collected for measles Summary tables for the activities of the 1st round data collection A. Secondary data collection Region Status Zone Status Wereda Status Status Complete d Follow up Complete d Follow up Tigray Follow up 4 Afar region Follow up 2 Amhara Follow up 4 4 4 Oromia region Follow up 3 2 5 Somalia Completed 2 SNNPR Completed 2 1 5 Addis Ababa Follow up 1 5 2 12 16 14 2 B. Retrospective primary data collection completed Region Status Focus Group Discussion ( FGD) Key Informants Household Tigray 8 90 800 Afar region 5 29 400 Amhara 18 170 1,200 Oromia region 19 139 1,200 SNNPR 20 160 1,200 Somalia 5 40 400 Addis Ababa 10 100 800 85 728 7,200 C. General and Specific Case Studies from six selected case study werdas. Sr. I. General Case Study II. Specific Case Study No Samples No of persons Sr. No. Samples No. of Persons 1. Stool 1. Diarrhea a) SAF pres, (parasites) 3,211 a) SAF pres, (parasites) 135 52 b) Cary Blair trans.(Bact.) 2,346 b) Cary Blair trans.(Bact.) 134 c) Direct microscopy 3,206 2. TB: Septum 64 2. Malaria :Blood film 2,194 3. Water sample 39 3. Measles Typhus: Serum 3,042 4. Blood: RF 56 5. ARI-Ames Trans. Med 29 6. Blood :hemoglobin 1,056 7. Anthropometric measurement 1,056 Table Financial Report Sr. No Birr 1. Total cost of the project- 1,000,000.00 2. Cash transferred to date to EHNRI 700,000.00 3. Expenditure to date by EHNRI 591,925.94 4. Remaining balance at EHNRI 108,074.06 5. Total remaining project balance 408,074.06 6. Required cost to complete the remaining work by EHNRI 501,330.00 7. Difference to be asked as supplementary 93,256.00 Major constrains encountered The major constrains faced under this the research study of Vulnerability to Epidemics include: 1. Rainfall started early around May: The first round data collection was planned in May and June 2001. This year the rain unusually starred early in May in most of parts of the country. There was a problem to reach the selected PAs with all the laboratory equipment during the general case study. Some groups have to use donkeys and assistance from the villages to carry the laboratory equipment. 2. Shortage of funds the budget allocated for the this project is said to be inadequate to cover all the four planned components ( secondary data collection, retrospective, general and specific case studies) 3. In availability of information as planned from surveillance reports and outpatient records on monthly basis for the last ten years 4. Inadequate cases for the different planned disease at time field visits 5. In accessibility to some of the PAs from which the retrospective and the general case studies planned to be collected from the three Pas that include the three agro￾ecological zones Measures suggested and recommendations 53 · Ask USAID through DPPC for supplementary fund of about Birr 93,350 to complete the VEPP project successfully · Proposal to screen HIV/AIDS: One of the successes story under this project is an adequate blood/serum sample collection from over 3000 persons from six selected case study werdas. This was over 100% achievement since the planned samples were only 2400 individuals. These samples are collected from the rural community of different direction where the magnitude of the prevalence of HIV/AIDS is unknown. Further screening of these samples for HIV/AIDS will clearly show the status of this diseases in the rural community for better planning and intervention for policy makers. We suggest preparing additional and separate proposal for this activity. Remaining Activities The major activity remaining to be undertaken are divided in five phases ? Phase I –July- August 2001: · Undertake specific case studies focused only on typhus. · Creation of a database for the retrospective data, the general study and about 9 specific database for the seven diseases, nutrition and TM · Analysis of the biological sample collected from the general and from the specific case studies. · Other activities like request for the final installment, literature review in different disciplines can start at this phase. ? Phase II- September- November 2001 a) Field activity focused on malaria Parasitology and entomology. b) The majority of the biological sample laboratory examination results will be entered in a computer and analysis of the data base from retrospective, general case and specific case except malaria would be completed ? Phase III- December 2001 : The major activity will be to finalize all data entry and analytical process, produce tables and graphs for the draft report. ? Phase IV- January 2002 : Produce draft report ? Phase V- February 2002: Prepare the workshop to disseminate the result