LIVELIHOODS INTEGRATION UNIT (LIU): EVALUATION OF LIVELIHOODS BASED NEEDS ASSESSMENT PILOT IN THE SNNP REGION April 2007 This publication was produced for review by the United States Agency for International Development. It was prepared for under contract no. 663-C-00-06-00420-00 by a consultant for The Food Economy Group, LLC. Livelihoods Integration Unit: Evaluation of Livelihoods Based Needs Assessment Pilot in the SNNP Region This study is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents of this bulletin are the responsibility of the consultant who conducted the evaluation for the Food Economy Group and do not necessarily reflect the views of USAID or the United States Government or the Food Economy Group. CONTENTS Acronyms Progress to Date: April 2007…………………………………………………………………………..i Executive Summary………………………………………………………………………...………...iv Section One: Introduction……………………………………………………………….………….I-1 Section Two: History of Early Warning Systems in Ethiopia………………………………...…II-1 What is an Early Warning System?....................................................................................II-1 The Need for Early Warning Systems in the horn of Africa……………………………II-2 Section Three: The SNNPR Baseline Pilot: Objectives, Outputs, and Process……..…………III-1 Objectives of baseline pilot……………………………………………………………….III-1 Baseline outputs and results…………………………………………………...…………III-1 Data collection and analysis………………………………………………….…………..III-3 Uses of baseline information………………………………………….…….…………..III-10 Baseline integration with early warning system………………………………………III-11 Section Four: Training, Capacity Building and Sustainability…………………..….………….IV-1 Training for the baseline study………………………………………….……………….IV-1 Training for scenario or outcome analysis…………………………….…….…………..IV-2 Training for the monitoring system………………………………………….…………..IV-3 Training for report writing………………………………………..……………………..IV-3 Training strategy for sustainability………………………………..…………………….IV-3 Section Five: Standards and Standardization………………………………...…………………..V-1 Section Six: Conclusions and Recommendations……………………………..…………………VI-1 References Annex A: Persons met Annex B: Consultant’s Scope of Work Annex C: Monitoring and Databases i ACRONYMS ADPC Asian Disaster Preparedness Center CIDA Canadian International Development Agency DFID Department for International Development (UK) DMISA Disaster Management Institute of Southern Africa DPPA Disaster Prevention and Preparedness Agency DPPB Disaster Prevention and Preparedness Bureau (regional) DPPC Disaster Prevention and Preparedness Commission EEW Ethiopian Early Warning EFSA Emergency Food Security Assessment EFSRA Emergency Food Security Reserve Administration ENA Emergency Needs Assessment EPSP Emergency Preparedness Strengthening Program EWWG Early Warning Working Group EWD Early Warning Department EWS Early Warning System FAO Food and Agricultural Organization FEWS NET Famine Early Warning System Network FSAU Food Security Analysis Unit FSCB Food Security Coordination Bureau GDFRE Government of the Democratic Federal Republic of Ethiopia GIS Geographic Information Systems HEA Household Economy Approach IR Intermediate Result LIU Livelihoods Integration Unit LIU SC Livelihoods Integration Unit Steering Committee MoARD Ministry of Agriculture and Rural Development MoH Ministry of Health MoWNR Ministry of Water and Natural Resources NAM Need Assessment Methodology NDPPC National Disaster Prevention and Preparedness Commission NSA Nutritional Surveillance Activities OCHA Office for the Coordination of Humanitarian Affairs PA Peasant Association PIO Public International Organization PPPD Policy, Planning, and Programs Department PSNP Productive Safety Net Program RDPPC Regional Disaster Prevention and Preparedness Commission SC/UK Save the Children / UK SNNPR Southern Nations, Nationalities and Peoples Region SO Strategic Objective SOAG Strategic Objective Agreement SPSS Statistical Package for the Social Sciences UNDP United Nations Development Program USAID United States Agency for International Development USG United States Government WDPPC Woreda Disaster Prevention and Preparedness Commission WFP World Food Program VAM Vulnerability, Analysis and Mapping Unit ZDPPC Zonal Disaster Prevention and Preparedness Commission The DPPC has been restructured and is now called the Disaster Preparedness and Prevention Agency (DPPA) working under the umbrella of Ministry of Agriculture and Rural Development (MoARD). ii PROGRESS TO DATE – APRIL 2007 This evaluation of the SNNPR pilot work has been useful in identifying issues the LIU needed to address, particularly in the areas of training, capacity building and staffing. It also provided insight into areas in which opinions on the way forward among key stakeholders diverged, prompting further useful discussions specifically related to monitoring and data storage/analysis. Some of these discussions are captured in Annex C. Current monitoring and data storage strategies being adopted by the LIU reflect the more detailed discussions that resulted from circulating the preliminary findings of this evaluation. A number of developments have been incorporated into the LIU workplan and design document as a result of feedback, both through this external evaluation process, and a preceding internal evaluation conducted by the LIU and through consultations with various stakeholders. These are grouped under the headings listed in the Executive Summary. Production & updating of livelihood zone maps The methodology used to update the maps has been improved. The livelihood zone map is drafted during the zoning workshop as usual. Each field team then carries copies of the PAs listed in their livelihood zone. These are verified at the woreda level to confirm that PAs have been assigned to the correct livelihood zone. In addition, changes to woreda boundaries (splitting or woredas and merging of woredas) are also noted. Results of the verification exercise are incorporated into the livelihood zone map during the baseline analysis and the resulting map is then finalized with the team members. The Tigray map was completed one week after the Tigray baseline fieldwork was completed and is now available from the LIU in Arcview shape file or JPEG format. The rapidity of this process has been facilitated by the arrival of the LIU database manager. Separate training strategies for baseline collection and monitoring. The emphasis in all of the training is on quality of data collection and on sustainability – i.e. identifying ways that the work can be carried forward in the future without compromising the quality. The emphasis is on inclusiveness – ensuring that capacity is built to various levels in a number of different organizations. Training process (baseline training, fieldwork, analysis) The training continues to be done by experienced HEA practitioners/facilitators who are supported by recently qualified HEA practitioners who mentor new trainees. The baseline training now incorporates additional group work and role play. These tools, coupled with mentoring both in the classroom and during the fieldwork and analysis, enables greater support to trainees. A trainer of trainers course has also been initiated to strengthen presentation skills of the baseline trainers. On￾going evaluations continue to feed into the development of the training process. In addition, team leaders’ skills have been strengthened to enable them to provide greater support to team members in the field. Other trainee support strategies currently being employed: ensuring that teams have 4 members (so that one member can cover each wealth group and the team leader can provide technical support). Time spent in the field collecting data has been increased by 25% to give teams more time to focus on cross checking data. Data quality and gaps Accuracy and cross checking continue to be emphasized during the baseline training and teams are encouraged to make use of the field handbook to cross check information collected. Analysis and cross-checking is led and facilitated by the team leaders on a daily basis. An interim analysis is completed half way through the field work to ensure that gaps are identified and plugged. This is followed by a comprehensive analysis at the end which includes a number of additional cross-checks. iii Team leaders now receive additional training in baseline analysis and the baseline spreadsheet along with a manual that has been developed to facilitate this work. The objective is to continue to increase the capacity of team leaders to support team members in the field. Key points from the training manual are now incorporated into the new team leaders’ field handbook. Gender Gender continues to be a central focus of the LIU – not only in terms of who is interviewed but also in terms of who is trained and the type of data collected. Consultations in Tigray have led to teams collecting information on the proportion of female-headed households. The livelihood zone profiles try to capture information on both female and male specific activities. Participation in the analysis. This continues to be emphasized by the LIU such that now there are several people who could facilitate either the seasonal assessment analysis or the baseline analysis. The LIU feels strongly that the more people who are able to use the data, the greater will be the uptake and utilization of the information available. With this in mind, a simple monitoring system is now being piloted in SNNPR – see below. Incentives to retain capacity within the DPPA The LIU together with USAID and the DPPA/DPPB’s are continuing to look into ways of keeping skills in-house. Options being reviewed at present include: offering internships (where trained staff provide mentoring to new trainees) and more formalized consultancies. However, additional suggestions within this topic are always welcome by the LIU. Sphere standards Nutrition surveys in Tigray are now being conducted by livelihood zone – which should contribute to improved interpretation of nutrition status data. The issue of thresholds is at the core of HEA and the work of the LIU – the survival threshold incorporates the sphere recommended intake of 2100 together with a small allowance to cover food preparation (salt, soap, kerosene). Further work still needs to be done to explore the desirability and options for incorporating sphere standards into the livelihood protection threshold. Non-food needs assessments For further discussions on this, see the Livelihood Integration Unit Information Sheet number 3 – which provides an overview of the contribution that HEA can make to non-food needs assessments. Monitoring The LIU is interested in initiating a monitoring system that is simple, monitors changes in the number of people in need, and can be interpreted at the woreda level. A pilot has been initiated in SNNPR and is described in the Livelihood Integration Unit Information Sheet Number 4 (May 2007). Whilst it is rather early in the day to evaluate this – it is likely that a major constraint will continue to be the issue of staff turnover and incentives. In addition, the LIU is looking at ways that HEA baselines can contribute to on-going development of the WFP/WB weather risk management project. Appendix C1 summarizes recent discussions on approaches to monitoring. On-going evaluations On-going evaluations have been incorporated into the training to ensure that participants (trainees, team leaders, other facilitators) feed into the development of the project. This has led to a number of practical modifications to the data collection tools – including where feasible: briefing woreda officials on the work, proposed dates of field visits, woreda data collection formats, and sending the woreda formats to the field in advance of the field teams visit. LIU APRIL 2006 iv EXECUTIVE SUMMARY The overall goal of the HEA pilot was to introduce a methodology that would enable the DPPA and its partners (at woreda, zonal, regional and federal levels) to improve their ability to respond in a timely and effective manner to predictable and unpredictable emergencies through an improved emergency needs assessment methodology.1 The more fundamental and challenging objectives of this initiative were to achieve: • Improved understanding by the DPPA and its partners of the livelihoods and coping strategies of populations vulnerable to hazards. • Transition from subjective methods of analysis to a more rigorous quantitative and analytical approach. Across the discussions held with a broad range of informants, there was evidence of a distinct appreciation of what HEA based analysis brings to Ethiopian Early Warning System among informants within DPPA as well as from the broader emergency assistance community. The HEA based approach represents a decisive improvement over the former system for several reasons: • Its livelihood profiling methodology has been developed and field-tested in many countries over a number of years and has gained international recognition and credibility with a wide range of authorities on early warning systems. The adoption of the HEA approach is bound to reassure the international donor community of Ethiopia’s intention to deliver transparent and defensible estimates in future appeals for humanitarian aid. • It has moved staff at all levels beyond the limits of a crop production framework, broadening its perspectives to consider a fuller range of factors that defines food security in terms of access to food in its varied dimensions for the household. • It offers a superior analytical tool that can be used, potentially, to capture variations between acute and chronic vulnerability and between relief and development needs. • It is community-focused and provides an objective vehicle of communication and information flow between community and government. Moreover, its community focus allows it capture a more detailed livelihood profile that can serve as a basis for development interventions. • Its uses, actual and potential, include at least three critical areas: emergency response, distinction between acute and chronic vulnerability, and development response. The focus of this evaluation is specifically on the emergency response. The critical question among the various respondents was not whether an alternative should be considered, but whether certain aspects of the methodology and its introduction might be improved, thus rendering it more robust within an environment of constraints and practical challenges. The current evaluation was charged with identifying areas that require further strengthening and focus in the future, thus providing guidance to the expansion of the livelihoods-based needs assessment system. Some of the more salient recommendations are summarized below. More detailed recommendations and suggestions can be found in the respective sections and in section VI. Production and updating of livelihood zoning maps. The livelihood zoning maps have clearly enriched Ethiopia’s capacity to present the situation of its population to the world. However, the cost of producing these maps individually, as they have been produced during the pilot phase, makes the process difficult to sustain in the future. The time and cost of map preparation was not only due to the reconciliation of three different sources of information, but to the manual nature of the map production process. A more appropriate method of producing livelihood zone maps needs to be developed by the LIU. 1 The “predictable” part is largely addressed by the Productive Safety Net Program (PSNP) of the Food Security Coordination Bureau (FSCB). v Separate training strategies for baseline data collection and monitoring. Training for the collection of baseline data should focus on data quality first, then participation and sustainability; training for monitoring and system maintenance should focus on participation and sustainability, improving data quality with good practice and procedures over time. A number of specific recommendations with regard to training are offered in Section IV. Training process. The training process needs to be improved so that it is clear at the end of training who “got it” and who didn’t. One day of field practice is not usually sufficient for close observation, nor do written tests taken at the end of training period provide reliable performance indicators. The systematic and well-planned use of role play throughout the training period can help to determine levels of understanding and motivation and identify areas that require reinforcement. Data quality and gaps. The process by which data inputs are verified should be improved, especially during the first days of data collection. Verifications should be reinforced through spot checks by team leaders and in collaboration with PA level development agents. Gender. The contribution of women to household food security is virtually invisible in the profiles. According to interviewers, women were either absent or else did not participate in the discussions and thus their input is not included. This serious bias needs to be corrected through training and awareness building. Separate group sessions should be held with women, offering them a venue that is conducive to their participation and input. Rapidity should not come at the expense of data quality and completeness. Participation in analysis. While the procedure of data collection for the baseline was clearly a team effort involving staff at all levels, the HEA procedure for the seasonal assessment is perceived by many to be less participatory than the previous one because it takes place within a select group at the regional office. While it is not realistic to devolve all analysis tasks to the woreda level, a revised monitoring system should include procedures that facilitate preliminary analysis and discussions which can feed analysis at higher levels. Periodic visits from the regional level to participate in such discussions at selected woredas would further strengthen the spirit of participation and collaboration. Incentives to retain capacity within the DPPA. Keeping higher level skills in-house within the DPPA will be a challenge given the limited options for internal incentives (promotions, salary increases, etc.) within the DPPA. One alternative to consider is a system of secondment such as has been used in other departments and, with some success, in countries like Senegal, Niger and the DRC. Individuals having a special set of skills that is in demand by government partners or other external agencies can be seconded on a contractual short-term basis, providing a source of additional income and prestige to the expert. Sphere standards. With regard to the Sphere recommendations for food security and needs assessment, the SNNPR pilot is well aligned with these, albeit with the following two exceptions: a) Women’s input should be explicitly included, “as women usually assume overall responsibility for food in the household and because they are the major recipients of food aid, it is important to encourage their participation in the design and implementation of programs wherever possible” (Sphere Handbook, 2004). b) Food security assessments must try to link up with information on the nutrition situation for “consideration of the impact of food insecurity on the nutrition situation is an essential part of food security assessment” (Sphere Handbook, 2004). Non-food needs assessments. In as much as non-food response is defined as being aimed at directly increasing food access for food insecure households, the HEA approach and methodology is clearly better suited to provide decision makers with relevant information than prior approaches. Its baseline data collection format base provides for a broad range of information that can potentially guide non￾foods needs assessments. Most of this information is qualitative, but certain elements are currently integrated into the spreadsheet. Moreover, the baseline can provide quantified expenditure patterns by vi wealth group to determine expenditure deficits. The data also provides a basis for determining the key parameters that are to be included in a revised monitoring system. Monitoring. Revision of the EWD monitoring system must include a clear framework for analysis of the information collected as well as being both practical and realistic. Suggestions from informants included: Collecting information at the PA level from randomly selected PAs (a more decentralized/bottom-up system) and/or collecting data on key parameters identified during the baseline work at the woreda level. Note: the pilot did not link on-going monitoring to the baselines – nor was the present EWD monitoring system evaluated. However, discussions by the evaluator on possible monitoring in the future led to the preceding suggestions. Woreda level workload. Through training, carefully selected monitoring software and a rational division of labor at the woreda level, the work load can be kept at manageable levels even while improving the quality and scope of information. Data entry and cleaning tasks should be well structured and simple enough so they become routine for clerical staff, i.e., levels where mobility is (possibly) less likely, especially if women are employed. An appropriate software can facilitate preliminary analyses that can be managed by DPPA staff at the woreda level. I-1 SECTION ONE: INTRODUCTION This report is based on an evaluation conducted from 22 September – 18 October 2006 and included 5 days in SNNPR. The Livelihoods Integration Unit (LIU) had already passed the planning phase and had started training and implementation activities in Tigray as part of its baseline extension program. Unanticipated changes in the project schedule had led to the rescheduling of the evaluation, such that it took place concurrently with the LIU’s start up activities. Still, the results of this evaluation are expected to inform the final design of the program. Background. The creation of an early warning system (EWS)2 in Ethiopia goes back to the aftermath of the country’s famine of 1973-74 which makes it the oldest on the African continent. The original version of the current system was created in 1977 with the support of the UN Food and Agricultural Organization (FAO) and its Global Information and Early Warning System (GIEWS). The design of the DPPA’s current needs assessment methodology came out of a workshop sponsored by the WFP in collaboration with the government’s Disaster Prevention and Preparedness Commission (DPPC) in 1996. It includes monthly reports on crop and climate conditions, seasonal food situation assessments and special investigations of selected problem areas. The results of this process have been repeatedly put into question by donors who have expressed concern over the reliability of the estimates, claiming in some instances that they were politically motivated. The Household Economy Approach (HEA) as a framework for analysis emerged from methods developed for helicopter surveys in 1991-92 by Save the Children UK (SCUK). It has since then been used by the organization in a number of food security surveys conducted in Ethiopia, including a pilot for a food security monitoring system in the Somali region and another in the Amhara region. Inspired by Amartya Sen’s entitlement theory, the approach sees food insecurity as a problem of access to food rather than simply a problem of food availability or supply. The new methodology was not accepted by the DPPC until in 2004 when an initiative to introduce it was funded by the United States Agency for International Development (USAID) through the DPPC’s Emergency Preparedness Support Program (EPSP). This initiative began with a pilot of livelihoods baseline studies based on the HEA approach in the SNNP Region. While overall lead coordination of the EPSP from the DPPA side is the Policy Plan and Programs Department (PPPD), the Early Warning Department is responsible for the technical implementation and management of the SNNPR pilot, with technical assistance from Ethiopia’s Famine Early Warning Systems Network (FEWS NET). The pilot was implemented over a period of approximately nine months, from late January to September 2005, with an extension of three months added to test the use of the baseline information for seasonal assessment purposes. It involved training federal and regional officials on the household economy baseline methodology, the identification and mapping of livelihood zones, the collection of data, the development of livelihood zone profiles, and the use of the baseline information in the DPPA’s seasonal assessment process during the meher pre-harvest assessment data in December 2005. Objectives of the Evaluation, proposed activities and methodology used. The SNNPR pilot implementation provided an important step in establishing a DPPA-led livelihoods-based needs assessment approach. This step must be seen within the broader context of past experience and future possibilities and options. The specific objectives set forth in the evaluation scope of work were to: • Review the successes of the pilot, to identify areas which require further strengthening and focus in the future, and provide guidance on the expansion of the livelihoods-based needs assessment system to other regions within Ethiopia (for example, enable the LIU/DPPA team to incorporate the lessons learned from the evaluation of the regional work into the design and development of 2 Used as the generic term, not to be confused with the FEWS structure created by USAID. I-2 the project). • Highlight aspects of the pilot exercises where standards (such as specific national or SPHERE) are especially important. • Initiate an iterative process of monitoring and evaluation that will be built in to the LIU’s management system in order to incorporate what works and discard what fails. Moreover, the evaluation should review the development of the baselines including usage of the baselines within the SNNPR – benefits of the approach should be highlighted together with lessons learned and implications for future programming. The evaluation findings/recommendations should be structured to feed into the LIU project objectives (and key result areas (KRAs) listed below. The more detailed information requirements are listed in the Scope of Work in Appendix B. The tasks whereby the information for the evaluation was obtained include: • A rapid desk review of available literature on early warning systems in the Horn, and on the reports which emanated from the Ethiopian pilot exercises (in SNNPR, Amhara, and Somali regions) • Consultations at the federal level with FEG, DPPA, USAID, LIU, SC-UK, FEWS NET, ACF, team leaders and monitors from the pilot exercises. • Interviews at the regional level team members and government staff at the regional, zonal and woreda levels There was a considerable amount and variety of written material to sift through, much of it in the form of internal documents such as proposals and reports, some as grey literature, some as published articles. The list of references provided at the end of this report refers to those sources that can be dated and attributed. A list of persons met is provided Annex A. II-1 SECTION TWO: HISTORY OF EARLY WARNING SYSTEMS (EWS) IN ETHIOPIA Ethiopia is undoubtedly one of the most experienced countries when it comes to the development of early warning systems and related information systems that aim to pre-empt the recurrence of large scale food emergencies. The issues surrounding the evolution of the current systems are complex and cannot be fully captured by this report even though they will undoubtedly influence the way forward. What is an Early Warning System? The basic purpose of an early warning system (EWS) is to forecast potential crises and provide timely notice when a food crisis threatens and, thus, elicit appropriate response. This requires a system of data collection that monitors people’s access to food (Davies et al., 1991). Traditionally, such systems have focused on the availability of food through agricultural production. More recently, the concept has been refined, based on the recognition that access to food involves more than just physical production. As Margaret Buchanan-Smith points out: it refers to a wide range of determinants of food security, taking the demand side into account as well as the supply side, and monitoring food entitlement, which may be measured through the use of a range of socioeconomic and other indicators …. an EWS should not be a process of data collection and analysis, which is regarded as an end in itself, but must be seen as part of a whole system which is geared to respond to food crisis and to prevent famine. This said, there is no single best model of an EWS (Margaret Buchanan-Smith, 1997). The Need for Early Warning Systems in the Horn of Africa While the Horn of Africa has considerable potential for economic development, it is a highly volatile area prone to both man-made and natural disasters. Conflict, within and between the countries of the Horn, is the most frequent man-made disaster that the Horn continues to face. To site the case of Ethiopia alone, it has endured internal conflict continually for more than three decades up until 1991. The war between Somalia and Ethiopia (1976), between Eritrea and Ethiopia (1998-00), and the disintegration of Somalia give further testimony of the extent to which the populations of the horn have been subjected to war-related disaster. When it comes to drought induced disaster, Ethiopia again stands out. Some of the most serious drought related disasters occurred in 1973/74, 1984/85, 1994/95, and again in 1999/2000 and 2002/03. Djibouti and Kenya are less well known for conflict related disasters but drought is a common occurrence in both countries. Moreover, there are indications that emergencies of both natural and man-made causes are increasing rather than diminishing in the area (Maxwell & Watkins, 2003). As the frequency and severity of such crises increases, lines between acute crisis and chronic vulnerability blur and response requirements become more difficult to analyze. The prognosis for the future is not cause for a great deal of optimism. The FEWS NET September 2006 report for the Horn warns that in spite of relatively good crop production conditions in crop growing areas this year, the situation in pastoral, agropastoral and marginal agricultural areas remains precarious. Moreover: if there is below normal rainfall, pastoral, agropastoral and marginal agricultural areas, particularly in Kenya, Somalia and Ethiopia, will face another season of poor production and will be pushed back into another major humanitarian crisis. Other factors including the results of peace negotiations in Somalia and Uganda, floods in Ethiopia, Somalia and Sudan, civil insecurity and crop and livestock diseases will also drive the food security situation in the region over the next six months (FEWS NET, September 2006). II-2 In a situation like this, it is imperative that vulnerable countries such as those of the Horn have a functioning early warning system that provides timely information and triggers timely response. The case of Ethiopia demonstrates the value of an early warning system. Although drought has become recurrent the and area and number of people affected increasing (reaching 14 million in 2002/03), the loss of life has decreased and this is attributed to the early warning system (Teshome, 2006). The Evolution of Early Warning Systems in Ethiopia The history of drought and famine in Ethiopia goes back to 250 B.C., with the most recent instances occurring in 2004-05. Although the amount of rain that Ethiopia receives is considered sufficient compared to other countries in Sub-Saharan Africa, its distribution and timeliness fails to provide a stable and reliable basis for production, and the country lacks appropriate technology for rainwater harvesting and irrigated agriculture. The Ethiopian Early Warning System (EEWS) is the oldest in Africa, established following the aftermath of the 1973/74 famine with the support of FAO/GIEWS. Its status was further enhanced in 1993 when the government issued a National Policy on Disaster Prevention and Preparedness. The policy has several implementation modalities. Early warning is the key component of the policy and the declaration of disaster is dependent on “convincing reports from the regional councils and the national early warning system” (NPDPM, 1993). The EEWS has been in a dilemma. The presence of strong government has meant that until recently, external agencies have been restrained from using their extensive knowledge and experience to the full in order to improve the system. More recently, there have been signs of improvement in the relation between these agencies and the government to the extent that the latter is prepared to incorporate new ideas and methodologies into its traditional assessments. The use of the Household Economy Approach in Somali Region as a basis for generating early warning information is one example, and the recent willingness to integrate livelihood approaches into the EWS another. Past techniques. Ethiopia’s early warning system is based on conventional techniques of monitoring food security indicators using four modalities: • Regular monitoring of key standard indicators from disaster prone woredas • Pre- and post-harvest assessments (used mainly in crop dependent areas) • Disaster area assessment (used mainly for assessing the impact of rapid onset disasters such as flooding) • Pastoral area assessment (used in pastoral areas) For the crop dependent areas the monthly monitoring is guided by the data collection format designed at the federal level which includes standard information relating mainly to crop production which are given a subjective classification in terms of comparison to ‘normal’ on a scale of 1-5 as illustrated below: 1. Much above normal 2. Above normal 3. Normal or near normal 4. Below normal 5. Much below normal Ideally, a well-designed monitoring system should provide sufficient information on the food security situation and have some predictive capacity. In the case of Ethiopia, the monitoring system is the weakest component of the EEWS due to, among other things, lack of commitment on the part of the woreda officials concerned, little or no incentives to complete the required formats, and poor II-3 communication systems that impede the flow of information. The monitoring system is therefore complemented by bi-annual inter-agency seasonal assessments conducted by multi-agency teams that travel from the capital to the regions. These teams carry out a pre-harvest belg assessment in June/July, and a pre-harvest meher assessment in October/November. Pastoral areas are assessed in June/July and again in December/January. In Somali region the meher assessment has been timed to coincide with the pastoral assessment and both are done in December. Further ad-hoc assessments are conducted when an emergency situation is reported, and in these instances, confirmation often comes through the use of nutritional assessment teams. In addition, FAO/WFP conducts its own food/crop supply assessment once a year (CFSAM). Macro-level data includes NDVI satellite imagery, rainfall estimates and climate prediction and is contributed by the national meteorological office and FEWS NET.3 Grain price information4 is linked to the grain marketing authority, previously funded by the EU Local Food Security Unit. It appears that there is no agency involved in livestock monitoring on a national basis. It is these seasonal assessments that define emergency requirements for Ethiopia. Following the pre harvest meher assessment, the annual appeal is normally launched in December/January. This appeal outlines the food and non food emergency requirements for the coming 12 months against which donors are requested to pledge resources. This is supposed to be reviewed later, following the post harvest assessment, when adjustments are to be made (inclusive of changes from the pastoral areas). However, such adjustments are rarely made and/or released to the Donor community. Instead, most adjustments come following the pre harvest belg assessment in July when a second appeal is released by the DPPA. To date, the EEWS has resisted the introduction of quantitative measurements of shocks and has relied on qualitative measures for a long time. For the system, it is not so much that x percent drop in rainfall reduces production by y percent that matters but people’s perception that rainfall is normal or below normal. The EWS in Ethiopia has also been criticized for being crop biased, paying too little attention to other sources of income including livestock in the pastoral communities. The authorities reject this assertion and blame the lack of a strong institutional setup in the pastoral areas. The introduction of the Household Economy Approach (HEA). Based on a livelihood security approach, the household economy approach (HEA) was initially developed by Save the Children UK (SCUK) in the early 90s and is now in use in many countries within Africa including Djibouti, Somalia, Tanzania, Burundi, DRC, Mozambique, Malawi, Zambia, Zimbabwe, Ethiopia, Chad, Liberia and Niger. The approach has also been used for needs assessment in a number of different countries.5 In Ethiopia, it was first introduced in the Amhara Region in 1995, but was not accepted as an instrument for early warning estimates at the time. Following the drought of 2000 in which the Somali Region suffered considerably, SCUK took on the mandate to establish an early warning system based on HEA in the Somali Region. More recently, it initiated HEA based EWS activities in the Afar Region. In 2005, the DPPC, with financial assistance from USAID, piloted the HEA-based methodology in the Southern Nations, Nationalities and People’s Region (SNNPR). After collecting baseline information in the 40 livelihood zones identified it used first the meher then the belg seasonal assessment to conduct its analysis and needs estimates. It is this pilot exercise that is the subject of the current evaluation. 3 The USAID-funded FEWS NET, established in the 1980s, is responsible for providing evidence based early warning information in the country. Its main role is to produce monthly food security reports using existing monitoring data of climatic and socio-economic indicators and livelihood information. 4 Market price monitoring focuses on crop supply areas only and is linked to the EFSR (Ethiopian Food Security Reserve). No market is monitored by the federal market monitoring system in Somali Region. Clearly this limits the usefulness of the information for early warning. 5 Views on this vary somewhat. According to some, HEA has been used for the production of livelihood baselines and profiles, but not for regular needs assessments in these countries. II-4 What is an HEA based methodology? The household food economy approach differs from the traditional early warning approaches used in Ethiopia in that it focuses on the access question of food security and not just availability. It tries to determine what households do to gain access to food during normal years and how they cope during bad years. The approach is designed to answer questions concerning food and livelihood security via the shortest reliable route, using a qualitative methodology for the collection of primary data and converting the information into quantitative proportions. FEWS NET provides a well-articulated summary of the approach and how it differs from the more conventional food security approaches: Thus the HEA based approach intends to address the following weaknesses of the current system: • Focus on availability (e.g., crop production) rather than household access • Poor links between assessment findings and response • Lack of credibility due to subjective processing of data The 3 basic steps of the HEA approach are: • Establish a baseline or snapshot of conditions during a relatively normal year (or other reference period), for households grouped by livelihood zone and wealth • Identify the problem based on actual or potential crisis situations • Calculate the effects of the problem and the possible reduction of the resulting deficit through coping mechanisms These three steps provide a basis for determining: • Types of assistance required (food as well as non-food) • Scale of the problem (from the size of the deficit) • Priority areas for assistance • Types of household most in need (for targeting) The principal data sources that the HEA system relies on include baseline data for the baseline assessment; hazard data for monitoring plus rapid assessment (2/year) data and response data for scenario analysis based on baseline plus hazard data. Local food security is often equated with agricultural production outcomes. Hence, a chronic or temporary production deficit against local food requirement is immediately translated into chronic or temporary food insecurity. Consequently most early warning and food security monitoring systems draw heavily from two information sources: (i) crop and/or livestock production data; and (ii) market price information. This is almost never the whole story. A full account of the ‘food economy’ addresses both food availability - that is, what food people produce—and food access—what cash people earn to purchase food. Data on casual employment or wild foods, or charity from relatives or the sale of handicrafts may be equally important to the livelihood story as data on crop and livestock production, and a knowledge of the relative importance of these can guide the design of more appropriate monitoring systems and better rapid emergency assessments. Using a livelihoods framework, we can inquire into household capacity to cope with stress, especially failed crop or livestock production; and we can appreciate household activities at different periods in the yearly cycle. All of which feeds directly into our analysis of need, helping to answer key questions such as; which areas and what types of household are likely to cope should a hazard strike and which will need assistance? What types of intervention will be most appropriate, and when and for how long should they be implemented? Thus for instance one could point to the position of poor households in a given geographical area who are highly dependent on urban employment. If urban employment declines, their labor will be less in demand: can they find alternative income elsewhere – and will they be competing with people from other zones in these activities? National officers working within their national early warning system have an immense knowledge of their countries. The livelihoods approach helps to provide a framework for the full use of that knowledge, as well as adding a new level of information to it (FEWS NET October 2004). II-5 The Outcome Analysis The principal steps that are used to establish a complete early warning information system are:6 1. Livelihood Zoning To identify homogenous population groups, sharing similar climatic and agro-ecological conditions and livelihood systems. This is done using a group process with representatives and informants from the various administrative zones. 2. Socio-economic / Wealth Breakdown To characterize the different wealth groups in each livelihood zone. This is done via group interviews with community representatives. 3. Analysis of Baseline Access to Food and Cash Income To quantify how typical households in each wealth group and livelihood zone obtain their food and cash income in a reference year, taking into account their food requirements, their expenditures and a range of coping mechanisms. 4. Analysis of Hazard To identify the hazard factors that are affecting households in the current year and quantify their economic impact [seasonal assessments] 5. Analysis of Response To investigate how households can respond to the hazard through expanding food and cash income options and switching expenditure from non-essential items to staple food [calculation based on HEA baseline data] 6. Analysis of Outcome To combine the baseline, hazard and response information into food security outcomes at household levels by wealth groups, livelihood zone and across the woreda. The basic formula is: Outcome = Baseline + Hazard + Response. In its approach to data collection, the HEA approach emphasizes triangulation through comparisons with secondary data or information from other informants and within each group interview through probing and, cross checking the responses to obtain a full household economy profile that adds up in terms of meeting the household’s basic food requirements. 6 Adapted from the FEG Guide to the Fieldwork. III-1 SECTION THREE: THE SNNPR BASELINE PILOT: OBJECTIVES, OUTPUTS AND PROCESS Sighting critical of weaknesses in the data collection and analysis processes of Ethiopia’s Early Warning System (EEWS), the prior head of the Emergency Preparedness Strengthening Program (EPSP) lobbied vigorously for piloting the HEA approach under the auspices of the DPPC. The pilot initiative involved the collection of baseline data in the SNNP Region, one of the most diversified regions of Ethiopia. The pilot began with an exhaustive mapping of the region in terms of key livelihood zones. The pilot exercise was funded by USAID and carried out under the FEWS NET contract. Objectives of baseline pilot The scope of work for the HEA baseline initiative lists three main objectives for the pilot project: • To map the SNNPR into “livelihood zones”, using these as basic units of analysis. Livelihood zoning groups people into geographic areas in which the majority of the population earns a living in similar ways and have similar access to markets. This is essential for any future outcome analysis. • To prepare baseline assessments (and profiles) for all 40 livelihood zones in the region, while ensuring that a core group of national staff are trained to properly use the information generated. • To conduct emergency assessments using the livelihood baseline information as reference point. The procedures “must be able to quantify household deficits and coping strategies in order to produce useful estimates of need on an annual basis and with periodic mid-season adjustments. Initially planned to take place from October 2004 to September 2005, the project was extended by three months to complete the third objective, with the following specific objectives for the extension: • To pilot test the improved livelihood-based methodology for undertaking emergency needs assessments using livelihood baseline studies as a reference point and to plan for the 2005/06 meher season needs assessment. • To provide technical support during the 2005/06 meher season needs assessment and to lead presentations of the results and implications of the study to different stakeholders at regional and federal levels. • To train two woreda officials from each woreda in the basics of the approach and in how to compile monitoring information for the problem specification. Baseline outputs and results7 The overall goal of the HEA pilot was to enable the DPPA and its partners (at woreda, zonal, regional and federal levels) to improve their ability to respond in a timely and effective manner to predictable and unpredictable emergencies through an improved emergency needs assessment methodology. This was to be accomplished through an improved understanding by DPPA and its partners of the livelihoods and coping strategies of populations vulnerable to hazards, and a transition from subjective methods of analysis to a more rigorous quantitative and analytical approach. These broader and results were to be attained through a series of specific outputs in information produced and in training achieved. The outputs delivered, listed below, are impressive both in quantity and in quality. 7 We distinguish between direct and concrete outputs and results that are longer term and often less tangible. III-2 Information and data outputs. The basis for an improved understanding of livelihoods and coping strategies is clearly visible across a prodigious number of information and data outputs: • Livelihood zone maps: mapping the region in terms of 40 identified livelihood patterns. • Livelihood zones profiles: providing summary descriptions of each livelihood pattern. These profiles include information on food, income and expenditure patterns, market access and trade flows, hazards and response strategies and wealth groupings that determine differences in access and coping strategies. • Analytical spreadsheet: includes both raw data for the zones as well as preliminary summaries and calculations and provide descriptive graphs • Integrated spreadsheet: permits the linking of various livelihood zones for analyses at zonal and woreda levels. • Woreda level reports: prepared according to a standard format and containing the new livelihood information • Regional summary report: providing the global profile for the region based on an improved understanding of livelihood patterns, experiences and vulnerabilities within the region. • A number of well-developed support and training documents, the results of many years of FEG experience in various countries. These will be compiled by the LIU and made available at regional, zonal and woreda levels. Training outputs Approximately 200-250 persons were trained from woreda level to zonal, regional and federal level. The training process included a variety of techniques and approaches, providing conceptual introductions, theoretical exercises, field practice and hands-on workshops. Structured training events: Number of Days Preliminary zoning workshop 3 Baseline training 5 Outcome analysis training 6 Woreda level training prior to seasonal assessments 2 Seasonal assessment training 2 Other hands-on training opportunities were available to participants through the baseline data collection process, the compilation and analysis of baseline information, writing of summary reports (by livelihood zone), the cleaning and entry of assessment data as well as participation in the scenario and outcome analysis. Broader results Initiatives that offer fundamental changes in perspectives and thinking take time to manifest themselves in tangible ways. What is evident from various discussions is that staff at all levels are moving beyond the limits of a crop production framework and broadening their perspectives to consider a fuller range of factors that define food security in terms of access to food in its varied dimensions for the household. The HEA based approach clearly offers a superior analytical tool for capturing variations between acute and chronic vulnerability and, potentially, between relief and development needs. It is community-focused and provides an objective vehicle of communication and information flow between community and government. Moreover, its community focus allows it to capture a more detailed livelihood profile that can serve as a basis for development interventions. Its uses, actual and potential, cover three critical areas: emergency response, distinction between acute and chronic vulnerability, and development response. These are described in greater detail in Section III, uses of baseline information. III-3 Data collection and analysis This is the center piece of the HEA innovation: a process that provides a more complete understanding of household livelihoods and coping strategies based on rigorous analysis that replaces the subjective and arbitrary processes of the past and delivers data in a timely manner with explicit implications for response programming. To do this, the approach categorizes the population into homogeneous sub-groups based on livelihood patterns and socio-economic standing, and queries the sub-groups using a Rapid Rural Assessment (RRA) methodology of qualitative inquiry within which basic procedures of household economic analysis are applied. Consistent with the RRA approach and qualitative inquiry in general, analysis is built into the very process of the inquiry, and the interviewer becomes not only reporter but also researcher. The division of labor between data collection and analysis is thus not as clearly separated as in more conventional survey methods. As has already been noted elsewhere, this type of methodology works best with a high level of skill, interest and motivation on the part of the interviewer / researcher. Based on the accounts of various informants and on interviews conducted in the region, such qualities are not commonly found among government staff. Can they be effectively transmitted given practical constraints? If so, how? If not, what are the consequences for the quality of the data? Below we look at each of the phases of the baseline data collection and analysis process to identify their strengths and weaknesses and to propose improvements where appropriate. Stratification of the population into subgroups. Analysis begins with the conceptual categorization of the region into livelihood zones in which households share similar agro-geological zones, climatic conditions, crop production patterns, and market access. Within these zones, representative communities are selected and within these a further breakdown into three or four wealth categories within which households share similar conditions related to access to resources. Methodologically, this is clearly one of its strengths, increasing the statistical efficiency of sampling. This approach provides not only a useful tool for capturing differences between zones and groups based on livelihood and wealth categories, but facilitates an in-depth analysis within each relatively homogeneous subgroups, enriching the level of detail and focusing the work of the interviewer/researcher. Livelihood zoning. Objectives: The result of this process is a livelihood zone map of some 40 livelihood zones within the SNNPR region. This is the basis of the livelihood profiles that describe the various production zones within which households share similar patterns of access to food production and markets. The process: The process of livelihood zoning in the SNNPR began with a workshop that brought together representatives from the zones and from some of the woredas. The output of the workshop was a preliminary demarcation of different livelihood zones in terms of availability of and access to food and markets (climate, geographic location, crop production, market access, etc.). Preliminary maps were produced and verified in the course of the field work for the collection of baseline data, and the modified information was reviewed with woreda experts and administrators until it received approval from the woreda. The output from this process was then submitted for the final production of maps at FEWS NET.8 8 The production of livelihood zone maps, from preliminary to final versions, was performed by the FEWS NET office in Addis Ababa, using GIS software. III-4 By all accounts, the process for producing the physical maps was very time-consuming. The main cause for the slow pace of the process was due to the problems of reconciling three sources of PA information: IN SNNPR, at the last census in 1994, there were 3900 PAs. In addition there were three sources of PA information that needed reconciling in order to prepare the maps: • the PA list prepared by the woredas and used by the teams to assign a livelihood zone code to each PA • the 1994 census, the only official source of population data that had to be used to estimate population by woreda and LZ • the digitized PA-level map, that had to be used to prepare the maps Difficulties reconciling the information were two-fold; • there were many differences in the spelling of PA names between the three lists, which made matching the lists very difficult • many of the PAs in the census list had been sub-divided since 1994 (so there were many new PAs in the woreda lists that could not be located within the census data), and there had also been many changes in woreda boundaries (with PAs being moved from one woreda to another, or older woredas being split into two). Geo referencing this data and using the Arcview software, individual shape maps were produced for each woreda as the verified and adjusted information was obtained. Thus some forty individual shape maps were prepared instead of one global shape file for the region that could then be broken down by woreda. According to FEWS NET staff who produced the final maps, the process as a whole was painstaking, taking them about a year to prepare the maps, from preliminary to final versions.9 The current SNNPR livelihood zone maps were prepared from a database of individual PAs, each of which was coded for the identified livelihood zone (instead of containing the data elements that permit the identification of livelihood zone). If a PA is administratively reassigned to another woreda, the livelihood profile of the woreda has to be adjusted and recoded manually. From their standpoint, the time and cost of map preparation was not only due to the reconciliation of three sources of information, but to the manual nature of the coding and map production process. One advantage of the process was that it was very participatory and thus served as a good introduction to the new methodology all the way down to the woreda level. However, despite the various verifications done in the field, several informants felt that the zone mapping process remained subjective and arbitrary and should be improved. The outcome: The mapping of 40 livelihood zones that resulted from this exercise is impressive and its colorful visibility signals clearly the advent of a new approach. It has been used by ACF (Action Contre la Faim) for its nutritional survey conducted in early 2006 and by an independent researcher associated with ACF. In both cases, the results were more pertinent and revealing due to the focus on livelihood zone rather than administrative district. Potentially, the map can be used for a wide range of purposes in the area of development as well as emergency assistance and planning. USAID supports the use of communication technologies at the DPPA, thus the outcomes and products of the HEA exercises have been posted on both the DPPA website (www.DPPA.gov.et) and the FEWS-NET website (www.few.net). Sustainability of process: The livelihood zoning maps have clearly enriched Ethiopia’s capacity to present the situation of its population to the world. Comments from informants indicate that the maps 9 In fact, the mapping information is still not fully compiled. Based on interviews with those that produced the maps: “The maps were never prepared using standard procedures. Rather we struggled to delineate a certain livelihood zone by asking, re-asking the regional offices and also verifying by using team leaders and informants at various levels, specially from woreda officials.” III-5 stimulate an ongoing discussion about the various livelihood zones suggesting that the concept has taken root. However, the cost of producing these maps individually, as they have been produced during the pilot phase, makes the process difficult to sustain in the future. As a first step, the development of a one shape file for all 40 LZs that facilitates updating and usage is recommended. For the future, the development of a PA based monitoring file that provides the basis for automated mapping should be considered. (However it should be noted that in SNNP region there are over 3000 PAs.) This consultant thinks that way of doing this might be to take a sample of PAs, create a database that includes the variables or parameters, obtained in a standardized manner, that go into the definition of a livelihood zone, and produce the maps automatically from the PA level database. This database could be populated so as to serve monitoring purposes as well. An added advantage would be that this database could be used to map zones defined for other purposes as well, such as: • Rapid and participatory food security assessments • Crop and livestock production surveys • Socio-economic surveys including Income and Expenditure surveys • Environmental impact assessment (suggested by WFP, 2003). Selection of communities (PA). For the collection of data from each livelihood zone, a sample of eight representative communities was selected per zone, based on discussions with, and recommendations of, woreda level informants. This process requires that woreda level informants are well-informed on conditions at the PA level. Since there is a lot of turnover at the woredas and since there are no systematic processes whereby past experience and knowledge are retained within the woreda knowledge base,10 the data obtained on PA level conditions could be limited to those that are known or are most readily accessible, thus biasing the sample to communities that are within easier reach of the woreda center. The baseline teams attempted to find the most longstanding and experienced members of staff to participate in the key informant interviews at woreda level and to assist in the selection of representative PAs to visit in order to avoid such biases. If a PA-level monitoring system existed, it might provide a more objective basis for choosing representative communities. Wealth ranking. Wealth ranking is another key feature of the HEA approach. Access to food and vulnerability to hazard and shocks is determined not only by the natural environment but also by social structures that distinguish between rich and poor households. The HEA approach uses focus groups with community representatives to identify households according to their social and economic standing within communities. Wealth ranking groups households into three or four wealth-based sub groups. Their relative size within the community is estimated through the use of proportional piling (using beans or rocks to represent size visually). The result of this exercise is used to find representative households from each group, to examine which groups are most likely to benefit from what kind of aid, and to estimate the number of beneficiaries per community and livelihood zone. Collection of primary data on household livelihoods The data at the PA level are collected by means of two sets of group interviews: the community interview and the wealth group interview. The community interview. The community interview is the first activity (after an initial round of introductions and presentations) of the team in a selected village or PA. Its purpose is to obtain general background information on the community’s past experience with hazards and shocks, prepare a seasonal calendar and develop a wealth ranking of households within the village. It is during this process that a reference year (one that represents a normal year within the past 5 years) is 10 Woreda level experts typically take their notebooks with them when they leave for another area: “they don’t leave anything behind”. III-6 established. Finally, the group exercise becomes the basis for arranging group interviews with the representatives of each of the identified wealth groups. It is during the community interview that the reference year, the last year that was considered “normal”, is determined. The reference year considered for the region as a whole was the agricultural year of 2003-04. This was used in all livelihood zones with one exception— the Hadero Ginger livelihood zone – in which ginger production that year was poor.11 During the community interview, timelines and seasonal calendars are developed. This includes information on the hunger period for the community as a whole. The community interview was conducted by the entire team, with members taking responsibility for different parts of the interview. The information obtained on crop and livestock production and markets was then used as a reference against which responses were checked during subsequent wealth group interviews. This interview was relatively straightforward and provided ample opportunity for cross-checking between different team members. The wealth group interview. The objective of the wealth group interview is to calculate the sum of ways in which an average household within each group obtains food during a normal year: how much they produce, buy, and receive through exchange or as gifts. Together with information on how much cash is earned and how much is spent to cover essentials and non-essentials, the information provides a relatively complete profile of household economies and the coping strategies used during difficult years. The rationale that underlies this interview is that a complete understanding of how households live during normal years, together with information on coping strategies helps to understand how they will cope when conditions deteriorate. The extrapolation of food and expenditure deficits, given identified hazards, can be summarized by the following equation: Baseline + hazard + response (coping mechanisms) = expected outcome. Responses from representatives of each wealth group within the community are obtained through semi-structured group interviews, facilitated by one single interviewer/researcher. The aim of each interview is to receive a group consensus on what the typical household produces, consumes, earns and spends during a normal year, as identified during the community interview. The interviewer is required to determine for the reference year and the average household within each wealth group: • How much food was produced, sold, exchanged, and consumed in terms of kilocalories and as percentage of total consumption needs (calculated at 2100 kilocalories per person/day WFP global standard). • Animals born, dead, sold, slaughtered and meat products sold, consumed, exchanged. • Days of lactation per animal type during each season and average milk production of which how much sold, consumed and exchanged. • All sources of food have to be itemized and transformed into kilocalories that are compared against minimum requirements per person, per day. The preliminary calculations, conducted during or immediately after the interviews, establish the extent to which the typical household meets its minimum caloric requirements based on 2100 kilo calories per person per day. Any food inputs cited are translated into kilocalories, whether they are produced or bought or received through other means. Interviewers are provided with checklists, reference sheets and calculators to determine how the typical household within each wealth group meets its calorific needs. The methodology requires instant calculations, checking and verification and the quality of the data depends on the skills of the interviewer. The strength of this approach is that its analysis goes beyond simple inputs based on food production to take into account the complete 11 Informants among interviewers were not in agreement on whether the reference year should be the same for the region as a whole or determined at the time of the community interview. According to trainers, it is now deemed more sensible to use the same year throughout the region. III-7 range of sources contributing to a household’s basic food requirements. An added advantage is the relative speed with which this information is collected. Weighing in against these strong points is the high level of skills required from interviewers that have to take on a researcher role, probing and cross checking complex interconnections as they go. The challenge is to meet the demand with the supply from a limited pool of available government workers – possibly reinforced by self-selected NGO staff. The work is exacting and tedious, requiring strong facilitation skills and a robust level of motivation, inquisitiveness, concentration and perseverance. Such skills are typically easier to find among university students with learning ambitions than among government workers. Since the baseline work is not continuous but takes place every 4-5 years, I would recommend arrangements with local universities to strengthen the pool of qualified candidates for this highly intensive work. Training. For the baseline study, the training began with a 5-day workshop during which the trainees were introduced to the conceptual underpinnings of the approach and provided with paper and pencil exercises to illustrate its logic. One day was dedicated to field practice in the vicinity of the training center, followed by group discussion. According to informants, there was no way of knowing at the end of training to what extent the material and practice had been understood by the trainees.12 The training together with the field work and baseline analysis is perceived as a process, some participants later complained that there was insufficient supervision during the field work. This varied between teams. In some zones where there were 4 wealth groups, the team leader often conducted interviews much like the other members and extra supervisory checks on others were up to the individual team leader. In livelihood zones with 3 wealth groups – this was less of a problem. At the end of the day, the team leaders entered the data submitted and conducted verifications of the internal consistency of the information. The team leader’s observations were shared with the other team members, typically the following morning since the entry and verification work often lasted well into the night. Use of secondary data The use of secondary data from documentary sources is, of course, desirable for this type of study, but is rarely available in an appropriately disaggregated form or user-friendly format. For the baseline study, the role of secondary data was limited to the market information obtained at the woreda level and other general estimates from subject experts. Information obtained from the community group interviews (overall crop production, etc.) was also used as reference information during the wealth group interviews (see below). The Baseline calculations. The approach is based on questions and probing to arrive at an understanding of how households survive taking into account various options for food access, cash income and expenditures. The method translates access to food, cash income and expenditures into common food equivalent units (kilo calories), a quantification that permits the comparison of different groups of people and areas. The interviewer is required to cross check responses as he receives them according to the guidelines below: Cross checking interview data: Checks can be made of both the internal and external consistency of the results. There are four main types of internal consistency check: • Food intake should, in most reference years, add up to at least 2,100 kilocalories per person per day • Income and expenditure should be approximately equal • Trends across wealth groups should pass a test of ‘reasonableness’ - does total production increase with wealth group? - does income increase with wealth group? - does the percentage of off-farm versus on-farm income change consistently across wealth group? 12 According to an LIU informant, there were instances where trainers considered that some trainees were quite capable of managing interviews on their own and did not require as many supervisory visits as other trainees. III-8 - does the proportion of expenditure on staple food decrease with increasing wealth? • Supply and demand factors should be consistent. Examples include: - number of days of agricultural labor ‘sold’ by the poor versus the number of days ‘bought’ by the better-off - receipt of gifts by the poor verses gift-giving behavior of the better-off The main checks of external consistency are in relation to secondary source information. Some examples: • Crop yields: how do village-level yields compare with Ministry of Agriculture yields? • Household size: how do reported household sizes compare to census figures? Source: FEG Guide to Analysis The methodology relies on the process of crosschecking to ensure the accuracy and reliability of the information through the group interviews. The first line of defense is the interviewer’s own process of crosschecking during the interview. The team leader verifies if things add up internally and make sense. The second check point is to review the data and select the average profile, discarding extreme responses. According to the guidelines, raw figures can be verified in a number of ways (eg comparisons with data provided in the field handbook), and those that cannot be verified are flagged during the analysis. According to the informants, there was no process whereby the raw interview figures were verified due to lack of time. If production figures are underestimated, the role of coping mechanisms could be overestimated in order to make things add up. Certain groups (e.g., the Fulani in Niger) are extremely reluctant to revise figures even when it is pointed out that they don’t add up. Getting things to add up is considered the interviewer’s problem and not their own. Having the interviewer work jointly with a development agent from the PA to conduct the interviews would help to verify the data on the spot. A second line of defense is to exclude interviews where things don’t add up and variables that are considered outliers (i.e., extremes in relation to either the mean of the remaining figures or in relation to reference values e.g. herd composition). The remaining data is based on a reduced sample that may or may not be representative of group experience during a given reference year. There was concern about the small sample size that remained once interviews and/or data points had been eliminated. Emphasis on triangulation is another feature that is expected to reinforce data quality. Since the availability of secondary data for triangulation purposes is limited, the process must rely on the information obtained directly from different informants. How does one decide which informant is closer to the truth? General perception puts the credibility of woreda informants at the low end of the scale and the credibility of PA agents at the high end. Community and wealth group informants fall somewhere in between. Gender. The guidelines emphasize the importance of different perspectives over sample size: “you can't rely on numbers alone to insure a diversity of views”. Thus the instructions encourage interviewers to make sure that they interact with individuals or groups having different perspectives depending on the topic. The most obvious differentiation is, of course, that between men and women. Women have a central role in all aspects of food security, especially where food consumption and expenditures are concerned. They also contribute directly to household income mainly through the sale of their labor, the sale of chickens and eggs and the sale of handicrafts and in other ways. However, there is no evidence that women were given a voice in the interviews. According to some informants, women either did not come to the groups or, if they came, did not participate actively. Creating separate groups for women was considered impractical because of time limitations. Thus the data represent the perspectives and inputs of men only. As has been found in many other studies, information provided by men on expenditures, incomes and consumption can be quite different, not to say less reliable, than that of women. Thus the absence of women represents a distinct risk to the quality of the data not only in terms of representing different perspectives, but also in terms of getting more accurate information on certain data such as expenditures. III-9 Data verification and entry into single zone spreadsheets The data sheets from each wealth group interview are collected by the team leader at the end of the day. The data are verified for inconsistencies and gaps and entered into the single zone storage spreadsheet. In exceptional cases, interviews that are considered unsatisfactory are discarded. Reasons for excluding an interview are discrepancies between food intake reported and the standard minimum food requirements, and between income and expenditures. The interviewer’s personal perception about the quality of the information plays a role as well. According to the perception of one interviewer encountered in the region, none of the groups categorized as poor or very poor told the truth (i.e. need would tend to be overestimated by this method). In that case, does the process revert back to the old ways of making the data “look good”, that is, in this case, “add up”? The data collection and verification process does not protect against this possibility. Data quality The HEA methodology offers an improvement on the purely subjective process of estimation on which the prior system relied. It introduces a strongly analytical approach to the collection of the data, while using a qualitative methodology based on probing that intends to capture a fuller and more complete range of livelihood experience, well beyond those limited to production and market activities that have dominated past early warning profiles. When the procedure works well, the economic logic applied to these inputs situates them in an equilibrated totality where things add up rationally as well as intuitively, producing a good and believable story. Should the data inputs, which are entirely controlled by the interviewer, be in doubt, there is no process that provides for verification at that level. While the training agenda dedicated a fair amount of time to a discussion of triangulation and bias, there is no opportunity provided for observing the interviewer to see how he interprets recommendations into actual practice. The fact that women’s activities and contributions are barely visible in the profiles provides suggest the possibility of data gaps that escape analysis. None of the informants could claim that they included the voices of women, even though women represent about 50% of the population and contribute possibly more to household livelihoods. This omission is especially critical where the poorer households are concerned. Women provide an enormous amount of labor against the generation of income in such households, as do sometimes children. When the workload of the woman rises, the health of small children often suffers. The effects are manifested by rising rates of malnutrition and morbidity. It has been observed that even in relatively well-off households and during relatively good years, a rise in women’s market activities can bring with it a rise in infant malnutrition. Another missing item is child labor. Poor households where children work instead of going to school compromise their future livelihood security. The ability to read and write strengthens coping mechanisms and options in a world that increasingly communicates through the written word. There are no direct questions included that ask household representatives from the different wealth groups for a description of the type and length of the hunger period endured during the year of reference. Even if the economic data allow separate calculations to obtain estimates, it is recommended to obtain a subjective estimate from the population for triangulation purposes. Reducing the length of the hunger period for the more vulnerable households is considered an important indicator of development and should be included in the monitoring data base. In summary, the HEA process represents an enormous improvement in the understanding of livelihoods and coping mechanisms which is likely to improve as the concepts take root. Its strength is its analytical approach, but there is room for improvement were the quality of data inputs is concerned. With improvements in training strategy and in data verification, that weakness can be reduced. III-10 Management of data base The entire SNNPR data base is currently stored in five Excel spreadsheets. The software is thus being used to combine both data storage and analysis functions. A more efficient approach to managing and querying a data base of this size would be to separate the two functions by entering the data in a data base management software such as Access or other software created for this purpose, and conducting analysis and queries with the use of analytical software (e.g., SPSS). Uses of baseline information There are three critical areas for which the baseline information can be used: emergency response, distinction between acute and chronic vulnerability, and development response. Emergency response Establishing a baseline data base represents in and of itself a major improvement to the prior EW system which lacks any objective basis for comparison. Maxwell & Watkins (2003) provide a striking example of how the lack of baseline information crippled the analysis of early warning information during the food security crisis of 1999-2000 in Ethiopia and was at least partially to blame for a slow and inadequate response. Since its completion in the SNNPR in October 2005, the HEA baseline data base has been used for two seasonal assessments, the meher assessment in November/December 2005 and the belg assessment in June/July 2006. Since the agricultural year has been relatively normal, HEA based estimates were not strikingly different from parallel conventional estimates used, except for an inclination to estimate needs coverage for larger numbers of households during shorter periods. The baseline data provides more specific responses to the basic questions asked by decision makers concerning who, where, for how long. Coupled with a revised and strengthened monitoring system, the HEA baseline will be able to further refine responses about when and how much. Distinction between acute and chronic food insecurity Food insecurity occurs when households fail to secure access to sufficient food for an active and healthy life without risking long term loss of such access. This can occur through catastrophic events that disrupt availability of and/or access to food such as in cases of drought or pests, or as a result of long-term poverty. In the first case, we speak of acute food insecurity, in the second of chronic food insecurity. The two conditions are related; for example, it is the chronically food insecure who are the most vulnerable to shocks, while acute crises often engender chronic food insecurity. The ability to distinguish between instances of acute and chronic food insecurity within a given area will provide a better basis for appropriate response. Estimates of chronic food insecurity will need to consider the kinds of variables that a household economic analysis delivers, i.e. coping mechanisms whereby food is secured. While experts continue to grapple with measurable distinctions between acute and chronic food insecurity, we can point to one example where the livelihood zoning approach helped to reveal pockets of acute nutritional crisis within a broader context of chronic nutritional insecurity in the Sidama woreda. When ACF (Action Contre la Faim) conducted its nutritional survey across the Sidama woreda in 2006, it found a SAM (severe acute malnutrition) rate of 1.7% for the woreda as a whole, thus insufficient to raise flags of alarm. When it examined the data by livelihood zone, it found the much more alarming rate of 3.1% in Sidama’s coffee zone. The baseline information was also able to reveal the story behind the difference, a story that is unique to the coffee zone with its dependence on the market. While the woreda’s maize zone had done exceptionally well, households depending on the sale of labor in the coffee zone had been struck by crisis. The potential use of the baseline data for organizations involved in nutrition, health and other sectors is clear. III-11 Another potential use of the baseline is the adjustment of the official poverty line in accordance with updated conditions and by livelihood zone.13 Poverty is an indicator of chronic food insecurity and the refinement of criteria or cutoff points will support better estimates of its scope. Development response As has been noted by many observers, neither emergency response nor the Safety Nets Program (SNP) will contribute much to solving Ethiopia’s food security problem in the long run unless complemented by other development packages/interventions.14 The HEA approach provides a more complete understanding of rural household economics in Ethiopia, disaggregated by livelihood zones and by wealth grouping. Thus the possible uses of the HEA baseline information for development purposes are clear and were repeatedly mentioned by various informants during the evaluation. Moreover, the fact that its results are visible and accessible via the internet makes it an even more powerful source of information for development agencies. Baseline integration with Early Warning System The baseline data base is an addition to the existing EEW information system and needs to be integrated with its other two components, the monitoring and seasonal assessment data. Its importance as a critical component of the EW system is underscored by Maxwell and Watkins who provide the example of the 1998-2000 food security crisis as a case where the absence of baseline information on pastoral livelihoods led to the failure to anticipate the extent of the crisis in pastoral areas (Maxwell & Watkins, 2003). The need for a more complete humanitarian information system (HIS) becomes more evident as crises become regular occurrences and the interest in, and emphasis on, chronically insecure populations increases (Maxwell & Watkins, 2003). The success of the HEA depends on its use which in turn depends on the extent to which appropriate monitoring information can be combined with the baseline data to predict outcomes and monitor these predictions. Some steps have already been taken by piloting the use of the baseline for two seasonal assessment analyses. What remains is the revision of the monitoring system and its integration with the baseline data base into a complete information system that can support not only emergency response but also short- or medium-term development programming. Seasonal assessments Speaking of Ethiopia’s Emergency Needs Assessment practice, a recent study conducted for the WFP notes: “While Ethiopia has one of the longest histories of emergency assessment practice with relatively high expert input (including from WFP), current annual ENA practice is largely based on negotiations between assessment teams, government agencies and international agencies – a process that is open to wide-ranging biases at all stages: community, district, regional and national levels. This can lead to actual humanitarian needs being inflated or understated, depending on the interests and perspective of particular actors.” (Haan et al, June 2005). The seasonal emergency assessments conducted 2-3 times a year are basically a response to the weakness of the monitoring system. If the monitoring system were adequately strengthened, there would be no need for these routine assessments that involve a considerable cost in time and resources. According to one authority, an emergency needs assessment should be triggered and geographically 13 The national poverty line has been set at 1075 ETB / person / year (food & non-food basket), with no changes since 1995 (unpublished study conducted for Action Contre la Faim, 2006). 14 The Safety Nets Program operates parallel to Ethiopia’s EWS, targeting some 7.3 million people who are considered chronically food insecure with support through a mix of cash and food. III-12 focused on the basis of early warning monitoring, rather than conducted as a separate stand-alone activity (Maxwell & Watkins, 2003). As one observer commented: “too much emphasis is placed upon the outcomes of these multi agency assessments and not enough reliance is placed upon the EW system itself”. However as noted later in this document, the EWS suffers from a lack of credibility. To counter criticisms of previous approaches, the current assessment process is carried out by multi￾agency teams and is considered more open and transparent because of the presence of members of other agencies. But as before, the data are obtained from woreda staff and, aside from the discussions that take place between the assessment team and the woreda people, cannot be considered as objectively representative. The meher seasonal assessment of November/December 2005 provided the first opportunity to use the HEA baseline data for scenario analysis and the estimate of beneficiaries. The procedure is described in a subsequent report as follows: As was the case for prior assessments, the team obtained crop and other information from woreda staff, using basically the same checklist that had been used for prior assessments. The source of data inputs thus remains much the same as before as does the presumed practice of reworking the data (originating from the PAs) to fit with various agendas at the woreda level. According to informants, there are three sets of figures available at the woreda office for crop information: the figures intended for the regional office, the figures intended for the federal level and the figures considered “closest to the truth”. The cleaning, entry and analysis of the data was conducted at the regional office. The integrated spreadsheet that is used for the analysis was still under revision and the analysis was conducted with the help of the author of the spreadsheet. The results of the analyses were communicated back to the woreda by the regional office. There was no further opportunity to discuss these results at the woreda level. The information was integrated into the humanitarian appeal of January 2006 where the new methodology applied to the SNNPR assessment was noted. Since the harvests had been relatively good in all but two zones (Sidama and Guraghe) and there had been no major crisis in the region during the prior year, there was no notable modification of the safety net figures that were issued.15 The next occasion for piloting the analysis of assessment data came with the belg pre-harvest seasonal assessment conducted in June-July, 2006. The assessment results of this assessment estimate the food requirements for August - December 2006. The procedure used for the belg assessment was essentially the same as for the meher assessment, though participants took a more assertive role with the spreadsheet analysis while still provided with backup support. Prior to the assessment, another two-day training workshop was offered on HEA concepts and the methodology used for food need estimation during the belg season. The analysis and zonal level report preparation was performed at the regional office from July 10-14. The region was debriefed by the assessment team on the results of the analysis on July 15, and the regional office communicated 15 For the country as a whole, 1.7 million people requiring emergency assistance were projected in the January appeal, compared with the appeal's peak number of 2.6 million (FEWS NET, 2006). There were 15 participants in the meher assessment: 4 people from Federal DDPC, 3 Regional DPPB, 1 Regional FSB, 1 Federal MOA, 3 UN, 2 USAID and 1 NGO. The participants were split into 4 teams and were led by 2 Federal DPPA and 2 USAID team leaders. The teams visited 40 selected woredas and 5 special woredas and covered all food insecure parts of the SNNPR. Detailed discussions were held at zonal level for the woredas that were not visited. A separate analysis was conducted for each livelihood zone in each woreda using an integrated spreadsheet to facilitate the organisation of information and the speed of calculations. (Final Phase report, 2005) III-13 the results to the woreda staff. According to informants at the regional level, the new process as a whole makes it even more difficult to crosscheck the data at the woreda level. The recommendation is to ensure some discussion at the woreda level before the results are issued. This is not to return to the former system of negotiating estimates with woreda staff, but to promote transparency, participation and capacity building. Where future analysis is concerned, further training will be provided on the use of the integrated spreadsheet prior to the next meher assessment to selected persons, including a representative from the DPPA and from FEWS NET. Monitoring system As mentioned earlier, the current seasonal assessment process was established largely as a correction for the shortcomings of the EW monitoring system. The present DPPA EWD monitoring system is based on ranking, where data collectors compare the current information with what is considered “normal” and rank it accordingly. According to informants in the region, the monitoring data is typically from 3-10 days late, with 10-day delays considered normal. The delays are partly due to transportation, partly due to low capacity at the woreda level, staff turnover, 16 and the fact that there is no training schedule for newcomers. With the exception of the crop, rainfall, and market price data, much of the data collected by the monitoring system is not reported on a regular basis (see DPPA website for the monthly monitoring reports www.dppa.gov.et). The EEW monitoring system has been repeatedly criticized for its lack of credibility. In revising the monitoring system, this consultant feels that the LIU should aim to eliminate the need for routine bi-annual assessments (though not exclude the possibility for special inquiries). A well￾working monitoring system is preferable over a system relying on seasonal assessments not only because of cost issues but also because decisions that are delayed until the harvest season may not guarantee timely delivery of relief. Other indicators may warn of deteriorating food insecurity long before harvest assessment results are available, as was the case in Chad and Sudan in 1990-91 (Buchanan-Smith). Emergency needs assessments should be conducted when there is cause for special concern, based on alerts or flags provided by the monitoring system. The content of the monitoring system should include variables that: • Identify each PA in terms of its livelihood zone characteristics (i.e., crops, climate, seasonality, access to market, etc.) • Provide a basis for conducting seasonal assessments (crop and livestock conditions, etc.) • Can be used to forecast crop and livestock developments (inputs such as seeds, water, labor, etc.) Key parameters to be included (from baseline data and other sources) should be proposed, vetted and revised by a monitoring committee that includes representatives from different EWWG network partners, such as SC-UK, FEWS NET and MoARD. A simple framework for analysis and interpretation of monitoring data should be developed. Then, a standard format should be established that accommodates all location-specific variables and a regional coding structure should be developed for pre-coded variables. A simple data intake format should be developed simultaneously with the data entry format. Because of the large number of PAs in a given region (eg about 3900 in the SNNPR), the PAs to be included in the monitoring data base should be based on a 10-20% sample of all PAs, depending on diversity and resources. Based on the comments from various informants, the capacity to collect and submit reliable data is stronger at the PA level where extension agents are thoroughly familiar with local conditions. Once the monitoring format has been revised, PA level agents can be trained and charged with the collection of the data in PAs selected on the basis of a systematic random sample). 16 According to one informant, out of 13 woredas, 5 have received new staff over the past 6 months. III-14 A simplified and streamlined data entry process at the woreda level could be developed. The monitoring database can thus be used to create livelihood zone maps from the bottom up (see section on livelihood zoning). This consultant thinks that a well-designed monitoring system should: • Eliminate the need for routine seasonal assessments • Permit livelihoods-based analysis • Simplify updating of livelihood zones (due to woreda remapping, market changes, etc.) • Reduce cost of map production by digitizing location coordinates • Include built-in forecasting capability by including appropriate indicators (inputs, prices, production, etc.) • Contribute to non-foods analysis • Facilitate indicator trend analysis • Point to problem areas that require special inquiries or assessments To work out the specific details of the monitoring system, LIU should consult with FEWS NET and resource persons in Ethiopia, the US and other countries.17 Data management issues. The baseline results are currently stored in reports (profiles), single zone spreadsheets, and an integrated spreadsheet that permits analysis at the regional level, inking livelihood zones and woredas. The integrated spreadsheet is composed of six spreadsheets having each some 1860 columns. At the time of the evaluation (October 2006), there was only one person at the federal level that was able to navigate and understand this spreadsheet. The spreadsheet serves its purpose as an extension of the interviewer’s calculator, much as the calculator is an extension of the pencil and paper. It speeds up computations and provides immediate tabular and graphic output. It has recognized limitations as a data entry and data management software for large relational data bases, especially when compared against others that are specifically designed for this purpose.18 If appropriate computer resources, skills and funding are available other approaches should be explored. Based on various discussions and interviews, providing that appropriate computer and other resources (including training) are provided, the capacity to collect and submit reliable data is stronger at the PA level where extension agents are thoroughly familiar with local conditions. The data entry process at the woreda level can be simplified and streamlined with the help of an Access-based or similar software and programmed to provide output for preliminary analysis.19 The skills required for data entry and cleaning routines can be easily conferred to designated staff at the secretarial level, with supervision at a higher level where output and preliminary analysis is managed. PA level staff should be invited to monthly sessions where the preliminary output is reviewed. The content of the database should be transferable into SPSS (Statistical Package for the Social Sciences) as well as into GIS￾based software for global analyses at the regional and the federal level.20 A system that draws its information from the smallest unit, the PA, would also be in line with efforts to establish a more decentralized and bottom-up system. 17 Providing specific details for format and content is beyond the scope of this evaluation. 18 The recommendation of Microsoft is to use Excel for smaller flat files and to use a data base management program such as Access for large relational files such as the SNNPR data base. See “Using Access or Excel to manage your data” http://office.microsoft.com/en-us/help/HA010429181033.aspx 19 For example, the US Census Bureau has developed an excellent software for data entry and simple analysis that is available at no cost and that is already used in many African countries. 20 At the federal level, this task would be entirely manageable by certain staff at FEWS NET and by individuals such as Getachew Abate within the EWD. In the region, the evaluator identified at least one person with the potential capacity, albeit at the WFP office. The required software is already available and requires no further investment. III-15 Non-foods needs assessments In her report on risk and vulnerability in Ethiopia, Sue Lautze notes that “the persistence of disaster in the face of robust food aid response … should cause the government and the humanitarian community to analyze closely the assumptions that underpin the structure of early warning and disaster response in Ethiopia.” Specifically, the emphasis on food aid must be matched by strategies of non-food assistance to address the broader causes of malnutrition and mortality” (Lautze, 2003). In the face of chronic vulnerability and recurrent emergencies, the operational boundaries between “relief” and “development” have become increasingly blurred and appropriate interventions must be able to distinguish between “acute” and “chronic” vulnerability (Maxwell & Watkins, 2003). It is also noted that when non-food requirements are ignored in the face of food insecurity, livelihoods deteriorate more rapidly and emergency phases become more costly and require longer recovery periods (FEG, 2006). In as much as non-food response is defined as being aimed at directly increasing food access for food insecure households, the HEA approach and methodology is clearly better suited to provide decision makers with relevant information than prior approaches. Its baseline data collection format base provides for a broad range of information that can potentially guide non-foods needs assessments, including data on: • Livestock production inputs: livestock diseases; veterinary services and medicine; feed sources and cost, labor, seasonality, water (quantity and timing). • Crop production inputs: equipment and tools, traction, seeds, fertilizer, pest control, labor, number of seasons, extension services. • Water: sources (humans /livestock), quality, sanitation, quantity, labor (sphere standards). • Health: prevalence of diseases, type and access to infrastructure, type and cost of services, vaccination coverage, seasonality, mortality and morbidity over past year. • Nutrition: nutritional status data, other health surveillance data, type of and access to services, • Education: educational infrastructure, distance and access, cost of school materials, clothing, food, etc. Most of this information is qualitative, but certain elements are currently integrated into the spreadsheet as expenditures under the headings: fertilizer, seeds, livestock drugs, other essential and other general. Moreover, the baseline can provide quantified expenditure patterns by wealth group to determine expenditure deficits. The data also provides a basis for determining the key parameters that are to be included in a revised monitoring system. IV-1 SECTION FOUR: TRAINING, CAPACITY BUILDING AND SUSTAINABILITY Training and capacity building concerns are clearly not new to Ethiopia’s early warning system. Past efforts have invested considerably in training and building capacity within the DPPA. Most recently, between 1998 and 2006, the Institutional Support Project (ISP) implemented a capacity building program in collaboration with the DPPA. During this time, a number of EW system training manuals were produced, 806 participants were trained in disaster management, 1090 participants were trained in early warning, disaster management training was institutionalized in three institutions of higher learning in the Amhara region, and a professional association in disaster studies and management was established. In spite of these impressive achievements, the results of training were undermined by a chronic problem of high staff turnover. It was noted that experts in various government departments receive training in a given subject but fail to get the opportunity to put it into practice because of transfers to other departments within or outside the organization. Nor are there any mechanisms for handover or establishing an institutional memory at the woreda level. It was also felt that the context of decentralization rendered the task more difficult because it reduced the authority of the regional DPPB. Finally, there were problems of incentives among woreda staff that limited the effectiveness of the project. Considering the experiences just described, the LIU should: • Consolidate and analyze materials and experiences from past efforts. • Emphasize strategy over quantitative training outputs. • Target the most stable elements within the DPPA (for example, women are typically more stable than men, lower level staff tends to be more stable than upper level staff) for the institutionalization of EW system routines (data collection, data entry, preliminary analyses). • Attempt to engage staff from the MoARD, specifically at the PA and woreda levels. • Create more opportunities for the observed practical applications of concepts and techniques. • Apply techniques that help to identify persons needing further reinforcement prior to field application. Training for the baseline study Reasons for the reduced effectiveness of training in the past appear to be primarily strategic and structural, but there is also room for improvement where learning techniques are concerned. From the strategic standpoint, training should not only be about numbers (persons trained and certified) but should ensure that training be provided where it is well invested and where the returns match priority objectives. The main objective of the training for the baseline study is to obtain accurate, complete and reliable data that can be used as reference for early warning analyses and beneficiary assessment. Because the baseline studies are conducted only every 4-5 years, the baseline application skills as such are not likely to stick in a context of high staff turnover. That is not to deny that, in a more general sense, the skills and knowledge transmitted during training are extremely valuable in that they stress the importance of probing and of cross-checking. They are also useful in building a conceptual understanding of, and appreciation for, the more holistic approach the HEA offers among DPPA staff. However, if the priority is good quality data, then the quality of trainees matters, and we know that levels of interest within the available pool of government staff are very mixed. Therefore, we suggest that: IV-2 • Trainees be selected on the basis of interest, motivation and capacity, not only from a pool of government staff but also from students at local institutions of higher learning; and • More trainees be selected than are needed for the fieldwork (though care needs to be taken that the total group of participants do not become too large – preferably, it should not exceed 22-25) to increase motivation and ensure replacements if needed. This way, two critical objectives are met: the quality of data will be enhanced through the use of well-motivated interviewers, and the training will contribute to capacity building among government staff that are the most receptive and expresses an intrinsic interest in the process. Structural reasons such as high turnover and poor handover practices must obviously be dealt with by the appropriate administrative level. They cannot be corrected by training strategy or technique, no matter how well reasoned and prepared, and they will take time even if the political will for change is there, especially in a context of decentralization involving a lot of restructuring and staff movement. Based on a model of adult learning, training should always ensure a strong element of practice and simulation. Informants found the techniques used in the baseline training useful and interesting because the training applied a variety of presentation media in contrast to the more conventional lecture-based approach that trainees were familiar with. However, the element of practice and simulation can and should be further reinforced through well-structured and observed role play which can highlight difficulties that are not fully appreciated (or underestimated). The use of role play could also help to identify slow or poorly motivated learners. Training for the baseline study, in particular, emphasizes the importance of probing, crosschecking, inquisitiveness, in short, the skills and characteristics of a researcher. Training for this type of qualitative requires a rigorous mix of theory and practice, beginning with discussion and simulation (role play) in the classroom, followed up with observed field practices. When training is too short or lacks practice, the tendency will be to treat guides or checklists like questionnaires, with the interviewer falling back on coping mechanisms that will simplify the task by sticking strictly to what is listed on support documents, especially when the interview risks being long and tedious. The emphasis will be on getting things to “add up” rather than to ensure that all inputs are covered and not only the minimum required. The training period need not be necessarily lengthened, but the content of the program should be reviewed in terms of its specific mix of theory and practice. Spreadsheet and scenario analysis should be minimized at this point, so as not to overburden trainees with conceptual material. The baseline training agenda reviewed would appear to permit time for role play, and the FEG guide itself stresses the use of training exercises and practice in order to develop good interview techniques. However, informants had no recollection of using such practical exercises in the training environment prior to going into the field. Training for scenario or outcome analysis Analysis begins, of course, with the baseline interview process itself where the interviewer uses a calculator for basic calculations and cross checking. The data are then entered into a baseline storage spreadsheet from where summary graphs (bar and pie charts) are produced for each livelihood zone. This becomes the data basis for the livelihood profiles. Scenario analysis is currently done during seasonal assessments which are conducted 2-3 times a year. The field teams received two days of training on HEA concepts and the methodology to be used for food need estimation. The current analysis tool is the integrated spreadsheet that links the various livelihood zones across woredas and administrative zones. IV-3 Training for the monitoring system Training for the monitoring system should be less challenging because of the immediate opportunity for ongoing practice. It involves, more than anything, the learning of new procedures and routines, the acquisition of improved habits and the shedding of bad habits and practices. With the appropriate software, close follow-up and supervision, and the scheduling of regular collective review sessions, the transfer of good skills in managing a monitoring system is feasible, as has been demonstrated in other settings. Training for report writing The HEA based EW system constitutes a formal information system that relies on the written word as the principal means of communication. Moreover, since speed is an important factor, it must communicate directly with internal and external decision makers whose common language is English. The lack of good report writing skills in English was found to be a considerable hurdle to surmount during the baseline pilot, placing an enormous burden on the external consultants who had to rework all written materials submitted. A format was developed that would facilitate the task of summarizing results for field staff but the writing of the report itself fell to FEG consultants. LIU is in the process of developing a strategy to deal with this challenge and provide training that will improve in-country capacity (see LIU Work Plan, 2006). Training strategy for sustainability If training is to be sustainable, it has to be structured so as to protect it from staff movement and uncontrolled drain of skills and knowledge. A structure provides stability, permitting actors to move in and out with a minimum of disruption and loss of knowledge. Recommendations for a capacity building strategy are discussed below under three headings: baseline phase, monitoring system and incentives to retain skills within DPPA. Capacity building strategy for baseline phase and analysis The baseline work is a training ground for trainers, analysts and team leaders that are to carry the HEA logic, knowledge and practices into the future. Training and global analysis Master trainers, assistant trainers and trained analysts selected at federal, regional levels as well as among emergency/development partners and local consultants Formatted LZ profiles Team leaders with support from analysts & hired editors Profile mapping Consultants during transition; FEWS NET once the data base is digitized; then EWD/DPPA staff with support from FEWS NET and/or other agencies. Woreda/LZ level baseline analysis Team leaders and team members Data entry and cleaning Team leaders and team members PA level data collection Skilled interviewers supported by PA staff LIU has begun to train and certify future trainers and team leaders for the baseline work. It will be important that the certification process distinguish clearly between participation in a minimum number of training and practice events and demonstrated skill. This is what well-structured train-the￾trainer processes do. It should never be assumed that mere participation constitutes knowledge to avoid wrong habits and practices from becoming institutionalized. IV-4 A job description for trainers, analysts and team leaders will clarify what the demonstrated skills involve for each category. The three roles are not mutually exclusive where individuals are concerned, but they do require different sets of skills. Profile mapping will have to be contracted out until the data base is digitized and maps can be produced automatically and/or with relative ease. For the work of interviewers, it is recommended that the emphasis be on skill and that the pool of qualified candidates be enlarged to include students from local institutions of secondary education or higher learning (universities, institutes). Since this work takes place only every 4-5 years, the investment focus should be on data quality more than on staff training. Capacity building strategy for monitoring system The management of monitoring systems requires stable internal routines. Investment in capacity building for the operation and maintenance of the monitoring system needs to focus strictly on DPPA staff. Training needs to target different levels of expertise. Staff trained for global analysis can be drawn from the pool of internal staff trained during the baseline phase. Training of woreda level staff for analysis and data management can be facilitated by the appropriate software. Such software permits summary output and simple queries much as does the spreadsheet, but provides a more appropriate structure to manage a longitudinal database (consecutive time points or months) that can and should be geo-referenced and which facilitates the process of data entry and cleaning at the PA-woreda level. The proposed structure assigns the tasks of data collection to PA level staff. Global analysis Selected, trained staff at RDPPA and NDPPA Woreda level analysis and data base management Designated trained WDPP/EWD and/or MoARD staff Woreda level data entry and cleaning Trained clerical/secretarial staff PA level data collection PA-based extension agents (DAs) At each level, the tasks should be clearly described in a documented job description that remains with the structure rather than moves with individuals. Tasks and procedures should be standardized and documented. Systematic follow-up process will be needed until the new structures and procedures become routine. This follow up support could be provided by members of the trainer/analyst team trained during the baseline phase. Incentives to retain capacity within DPPA Keeping higher level skills in-house within the DPPA will be a challenge given the lack of a strong internal incentives program (promotions, salary increases, other). To compensate for the rigidity of the structure where incentives are concerned, a system of secondment could be introduced. Such systems are used with some success in countries like Senegal, Niger and the DRC. Individuals having a special set of skills that is in demand by government partners or other external agencies can be seconded on a contractual short-term basis, providing a source of additional income and prestige to the expert. Among the advantages of a system of secondment are that it: • Strengthens collaboration between govt. and external agencies • Promotes cross fertilization • Promotes alignment of objectives • Strengthens expertise through application to • Strengthens collaboration between government on various situations and issues • Rewards people for individual effort • Reduces demands on internal salary structure V-1 SECTION FIVE: STANDARDS AND STANDARDIZATION To institutionalize and fully integrate the livelihoods approach into the EEW system, national standards in food and non-food needs assessments should be reviewed and revised in line with Sphere standards. Sphere, an organization created in 1997, has developed minimum standards for humanitarian assistance agency, ensuring aid that respects the dignity of people, based on the humanitarian charter. With regard to food security and needs assessment, the SNNPR pilot itself is well aligned with Sphere recommendations, with two exceptions at this point: • Women’s input should be explicitly included, “as women usually assume overall responsibility for food in the household and because they are the major recipients of food aid, it is important to encourage their participation in the design and implementation of programs wherever possible” (Sphere Handbook, 2004). • Food security assessments must try to link up with information on the nutrition situation for “food insecurity is one of three underlying causes of malnutrition, and therefore wherever there is food insecurity there is risk of malnutrition, including micronutrient deficiencies. Consideration of the impact of food insecurity on the nutrition situation is an essential part of food security assessment. However, it should not be assumed that food insecurity is the sole cause of malnutrition, without considering possible health and care causal factors” (Sphere Handbook, 2004).21 Another area to consider is local emergency response systems. The baseline should include questions on systems at the community level and include information on these local systems in its reporting. Sphere has developed checklists based on adapted standards. This should be kept on hand for reference purposes whenever new baselines and assessments are initiated. The baselines are expected to contribute to the establishment of standard procedures for data collection and analysis; national standards for annual caloric requirements per person, food diversity and water consumption. Certain non-food categories require standards as well as non-food assessments gain in importance (production inputs, health expenditures, etc.). As the system as a whole is revised, there will be a need to develop standardized procedures and methods for data entry, storage and communication at all levels. This is especially so at the woreda level where there is a lack of proper recording and archival procedures. Issues of and approaches to procedural standardization is addressed in the previous section on training strategy. 21 In more practical and operational terms, outcomes from nutritional surveys could be imported into the PA monitoring data base on a quarterly or annual basis. VII-1 SECTION SIX: CONCLUSIONS AND RECOMMENDATIONS The overall goal of the HEA pilot was to introduce a methodology that would enable the DPPA and its partners (at woreda, zonal, regional and federal levels) to improve their ability to respond in a timely and effective manner to predictable and unpredictable emergencies through an improved emergency needs assessment methodology. The more fundamental and challenging objectives of this initiative were to achieve an improved understanding by DPPA and its partners of the livelihoods and coping strategies of populations vulnerable to hazards, and a transition from subjective methods of analysis to a more rigorous quantitative and analytical approach. Across the discussions held with a broad range of informants, there was evidence of a palpable appreciation of what HEA based analysis brings to EEWS across the spectrum of stake holders within DPPA and the larger emergency assistance and development support community. The HEA based approach represents a decisive improvement over the former system for several reasons. • Its methodology has been developed and used in many countries in the region, and has gained international recognition and credibility with a wide range of authorities on early warning systems. The adoption of the HEA approach is bound to reassure the international donor community of Ethiopia’s intention to deliver transparent and defensible estimates in future appeals for humanitarian aid. • It has moved staff at all levels beyond the overly simplistic crop production framework for the prediction of food security crises. It is broadening perspectives to consider a fuller range of factors that defines food security in terms of access to food in its varied dimensions for the household. • It is household and community-focused and provides an objective vehicle of communication and information flow between community and government. Moreover, its community focus allows it to capture a more detailed livelihood profile that can serve as a basis for development interventions. • Its uses, actual and potential, cover three critical areas: emergency response, distinction between acute and chronic vulnerability, and development response. • The critical question among the various respondents was not whether an alternative should be considered, but whether certain aspects of the methodology and its introduction might be improved, thus rendering it more robust within an environment of constraints and practical challenges. With regard to the points emphasized by USAID in the scope of work, they are to some extent integrated within the text of this report, but can be summarized as follows: • Timely completion of reports: The reports were completed in good time but at some cost. Writing skills in the field are such that all the reports coming from the field had to be rewritten by the FEG consultant. This involved an enormous work burden on a highly skilled professional and is clearly not a sustainable option in the future. As mentioned earlier, the LIU work plan tries to address this issue through training and the identification of local resource persons. • Livelihoods-based emergency needs assessment methodology considered workable; data and results considered representative of SNNPR: As this evaluation has pointed out, the needs assessment methodology is workable and the results representative of SNNPR livelihood profiles, but with the caveats presented below. VII-2 • Changed attitudes of government staff: There was a clear expression of interest in the new methodology at the regional level. At the woreda level, perceived attitudes tended to lean toward a wait-and-see perspective. Because of a high level of movement, staff are not certain to what degree the new methodology will influence their own work and, in particular, whether it will increase existing workloads. • Improved emergency programming: Since the pilot year was a relatively normal year, the new methodology’s contribution to emergency programming could not be explicitly tested. • Involvement of other line agencies: Line agencies within the government that contributed staff to the SNNPR pilot exercise include the following: Federal: EWC Secretary DPPA - NDPPA Regional: EWC Secretary RRB – RDPPA Zone: EWC Secretary RRD – ZDPPA Woreda: EWC Secretary MoA – WDPP & WDPPA • Official use of the livelihoods data in the “2006 Humanitarian Appeal”: The contribution of the new methodology based on household economy and livelihood based data analysis is officially acknowledged and described in the 2006 Appeal document. It confirms that for the SNNPR data “detailed analyses were completed for each livelihood zone in each woreda, and the results summed across livelihood zones to obtain a single woreda-level result” (2006 Humanitarian Appeal document). The points cited below summarize areas that can benefit from improvement. Accuracy of data inputs: While the process of data collection emphasizes cross checking and triangulation on the part of the interviewer, making things add up between food production, purchase and expenditure does not necessarily protect from inaccurate inputs. Recommendation: The process by which interviewers verify production estimates could be improved by engaging PA agents into the process, working in close collaboration with the interviewer. A well￾structured division of labor between the interviewer and the participating PA agent could alleviate the burden on the interviewer, speed up the process and reduce the tedium that appears to threaten the quality and the quantity of data. It would also serve to increase confidence in the data and a sense of ownership from the bottom up. Gender: The contribution of women to household food security is virtually invisible in the profiles. According to interviewers, women were either absent or else did not participate in the discussions and thus their input is not included. This serious bias needs to be corrected through training and awareness building. Recommendation: Critical gaps in the data threaten its quality and credibility. Separate group sessions should be held with women, offering them a venue that is conducive to their participation and input. Rapidity should not come at the expense of data quality and completeness. Production and updating of livelihood zoning maps: The livelihood zoning maps have clearly enriched Ethiopia’s capacity to present the situation of its population to the world. However, the cost of producing these maps individually, as they have been produced during the pilot phase, makes the process difficult to sustain in the future. Recommendation: Since the manual production and updating of the maps is likely to involve costs that are difficult to sustain in the future, the EWS data base should be structured to permit the geo￾referencing of location categories from the PA level upward. A revision of the data base is VII-3 recommended sooner rather than later, in order to reduce mapping costs during the extension of the baseline. Participation in analysis: While the procedure of data collection for the baseline was clearly a collective effort, the new procedure for the seasonal assessment is perceived to be less participatory because it takes place within a small group at the regional office. Recommendation: the new monitoring system should include procedures that facilitate preliminary analysis and discussions at the woreda level. Periodic visits from the regional level to participate in such discussions at selected woredas are recommended. Training process: While the materials and techniques used for training are of very good quality and generally appreciated, the process needs to be improved so that it is clear at the end of training who “got it” and who didn’t. Field practice is not usually a good method for the close observation that the process requires, nor are written tests taken at the end of training. It was generally expected that the training period continue into the initial data collection period, but no systematic measures were taken to provide the extra level of supervision that this would normally require. Recommendation: The training process needs to structured so that it is clear at the end of training who “got it” and who didn’t. One day of field practice is not usually sufficient for close observation, nor are written tests taken at the end of training good indicators. The systematic and well-planned use of role play throughout the training period can help to determine levels of understanding and motivation and identify areas that require reinforcement. Team leaders should use spot check procedures during the first 2-3 days of data collection. Selection of trainees for baseline work: Where the priority is good quality data, the quality of trainees matters, and we know that levels of interest within the available pool of government staff are very mixed. Recommendation: Where the recruitment of interviewers for the baseline study is concerned, trainees should be selected on the basis of interest, motivation and capacity, not only from a pool of government staff but also from students at local institutions of higher learning. More trainees should be selected than are needed for the fieldwork (though care needs to be taken that the total group of participants do not become too large – preferably, it should not exceed 22-25) to increase motivation and ensure replacements if needed. This way, two critical objectives are met: the quality of data will be enhanced through the use of well-motivated interviewers, and the training will contribute to capacity building among government staff that is the most receptive and expresses an intrinsic interest in the process. This also ensures a better return to investment within the DPPA. Training strategies for baseline and for monitoring: Different training strategies should be developed for the baseline work and for maintaining and using the monitoring system since priorities differ somewhat. Training for the collection of baseline data should focus on data quality first, then participation and sustainability; training for monitoring, assessments and system maintenance should focus on participation and sustainability, aiming to improve data quality with good practice and procedures over time. Non-food needs assessment: In as much as non-food response is defined as being aimed at directly increasing food access for food insecure households, the HEA approach and methodology is clearly better suited to provide decision makers with relevant information than prior approaches. Its baseline data collection format base provides for a broad range of information that can potentially guide non￾foods needs assessments. VII-4 Sphere standards: With regard to the Sphere recommendations for food security and needs assessment, the SNNPR pilot is well aligned with these, albeit with the following two exceptions: • Women’s input should be explicitly included, “as women usually assume overall responsibility for food in the household and because they are the major recipients of food aid, it is important to encourage their participation in the design and implementation of programs wherever possible” (Sphere Handbook, 2004). • Food security assessments must try to link up with information on the nutrition situation for “consideration of the impact of food insecurity on the nutrition situation is an essential part of food security assessment” (Sphere Handbook, 2004). Incentives for keeping higher level skills in-house within the DPPA: This is a challenge given the limited options for internal incentives (promotions, salary increases, other) within the DPPA. One alternative to consider is a system of secondment that has been used with some success in other departments and in countries like Senegal, Niger and the DRC. Individuals having a special set of skills that is in demand by government partners or other external agencies can be seconded on a contractual short-term basis, providing a source of additional income and prestige to the expert. There are several advantages to using a system of secondment that are listed in Section 4.5. Recommendations for a possible future monitoring system: Based on the comments from some informants, the capacity to collect and submit reliable data is stronger at the PA level where extension agents are thoroughly familiar with local conditions. Once the monitoring format has been revised, PA level agents can be trained and charged with the collection of the data in PAs selected on the basis of a systematic random sample. A system that draws its information from the smallest unit, the PA, would also be in line with efforts to establish a more decentralized and bottom-up system. Woreda level workload: The work load can be controlled and kept at manageable levels through a rational distribution of tasks at the woreda level. Data entry and cleaning tasks should be well structured and simple enough so they become routine for lower level staff where mobility is (possibly) less pronounced, especially if women are employed. R-1 REFERENCES Boudreau, Tanya and Coutts, Philippa, Food Economy in Situations of Chronic Political Instability. Overseas Development Institute, 2002. Boudreau, Tanya, The Food Economy Approach: A framework for understanding rural livelihoods. Relief and Rehabilitation Network (RRN), 1998. Buchanan-Smith, Margaret, What is a Famine Early Warning System? Can it Prevent Famine? Action Aid, 1997. Chemonics International with Food Economy Group. Livelihoods Baseline Study to Support Improvements in the Ethiopian Disaster Prevention and Preparedness. Chemonics International, Task Order Proposal for Livelihoods Baseline Study to Support Improvements in DPPA Emergency Needs Assessment Methodology, November 2004. Chemonics International, Commission’s Emergency Needs Assessment Methodology: A Task Order Proposal under the FEWS NET IQC, November 2004. Desta, Tedla/Save the Children – UK, Evaluation of the Food Security Monitoring and Early Warning Project – Somali Region, November 2004. Devereux, Stephen, Vulnerable Livelihoods in Somali Region, Ethiopia, Institute of Development Studies, April 2006. DPPA/USAID Emergency Preparedness Strengthening Program, Evaluation of 2004/05 Work Plan, November 2005. FAO, The elimination of food insecurity in the Horn of Africa: A strategy for concerted government and UN agency action, Technical Cooperation Department, 2000. Food Economy Group, Guide for Rapid Food Assessments, March 2004. Food Economy Group, Livelihood Zoning Guide, January 2006. Food Economy Group, Guide to the Fieldwork, (draft, no date). Food Economy Group, The Food Economy Spreadsheet: A Training Manual, 2004 Haan, Nicholas; Majid, Nisar & Darcy, James, A Review of Emergency Food Assessment Practice in Ethiopia, Overseas Development Institute for the World Food Program, June 2005. Jaspars, Susanne and Shoham, Jeremy, A Critical Review of Approaches to Assessing and Monitoring Livelihoods in Situations of Chronic Conflict and Political Instability, Overseas Development Institute, 2002. Lautze, Sue et al. Risk and Vulnerability in Ethiopia: Learning from the Past, Responding to the Present, Preparing for the Future, Report prepared for USAID, June 2003. Lautze, Sue; Aklilu, Jacob; Boyd, Erin, Assessments and Appeals: Strengthening Non-Food Emergency Responses in Ethiopia, The Livelihoods Program, September 2005. R-2 Lawrence, Mark. Household Economy Outcome Analysis: A Guide for the Food Assessment Unit, Somalia, FEG, March 2006. Lawrence, Mark and King, Alex. FEG. Guide for Rapid Food Assessments, March 2004. Levine, Simon and Chastre, Claire, Missing the Point: An analysis of food security interventions in the Great Lakes, Humanitarian Practice Network, ODI, July 2004. Livelihoods Integration Unit, SNNPR Feedback Workshop: Notes. September 2006. Maxwell, Daniel and Watkins, Ben. Humanitarian Information Systems and Emergencies in the Greater Horn of Africa: Logical Components and Logical Linkages, Overseas Development Institute, 2003. Montani, Abigail; Majid, Nisar. Conducive Conditions: Livelihood Interventions in Southern Somalia. Overseas Development Institute. December 2002. (incomplete!) Save the Children – UK. Report on Technical Review of Phase I of Project Improving Food Security Monitoring and Early Warning Capacity within the Regional Disaster Prevention and Preparedness Bureau-Somali Regional State. June 2003. SC-UK, DPPA & FEWS-Net. Sekota and Bugna Woredas, Woina Dega Livelihood Zone: Pilot Needs Assessment. April 2004. Shoham, Jeremy. Food Security Information Systems Supported by Save the Children UK: A Review, SCUK, 2005. Sphere Project. Food Security and Nutrition Assessment and Analysis, Sphere Handbook, 2004. Thompson, Larry & Winer, Nicholas. Ethiopia: The Struggle for Food Security. RI Bulletin. November 30, 2004. United Nations Consolidated Appeals Process (CAP), 2006 Humanitarian Appeal for Ethiopia. January 2006. USAID/Ethiopia. Request for Task Order Proposal (RFTOP) for Livelihoods Baseline Study to Support Improvements in the DPPA Emergency Needs Assessment Methodology. 2004. Watkins, Ben. Livelihood Zones: A Discussion of Basic Principles. A draft discussion paper prepared for the WFP. 2003. A-1 ANNEX A: PERSONS MET, INTERVIEWEES AND INFORMANTS Getachew Abate, EWD, DPPA Alemu Asfaw, Country Representative, FEWS-NET Ethiopia Nigist Biru, Deputy Country Representative, FEWS-NET Ethiopia Julian Chalimbaud, ACF Teshome Erkineh, Head of EWD, DPPA Demeke Eshete, SCUK Girma Haile-Michael, PPPD, DPPA Julius Holt, FEG Partner and Technical Consultant Alexandra King, FEG Partner and Technical Consultant Hailu Gebremichael, HEA Consultant, pilot team leader Haile Kiros Desta, HEA Consultant, pilot team leader Mark Lawrence, FEG Partner and Technical Consultant Yacob Loha, Head of Aid Coordination Team, Awasa Jane MacAskill, LIU Chief of Party Nancy Mutunga, Representative for FEWS-NET Kenya Steve Perry, former EPSP Coordinator Tewodros Yeshiwork, USAID/ALT Office, Senior Program Monitor, Tigist Yifru, CTO, USAID Fanaye, USAID/ALT Office, Program Monitor, Cassandra Chapman, PACT, formerly with SCUK Dessalign Teshoma, Head of Regional EWD, Awasa Teshome Dessalign, Field Monitor for WFP, Awasa Dakiye Clise, FS & DPPA Desk Head, Gamo Gofa zone Awlachew Kibebe Tschary, EW Expert, Gamo Gofa zone Bizuneh Glmedhin, Head of FS & DPPD, Walayta zone Mengisto Assfa, Head of FS and DPP Desk Getahur Aygaw, EW expert, KT zone B-1 ANNEX B: CONSULTANT’S SCOPE OF WORK This SOW has been compiled from information in the both the LIU proposal and the contract as well as from inputs from consultations with the LIU Steering Committee (12 September 2006) and the Internal SNNPR Feedback workshop (14 September 2006) Background to the pilot in the SNNPR. USAID has a long and close relationship with the GFDRE, and in particular with the DPPA, in providing support to early warning and livelihoods-support activities. In its Integrated Strategic Plan (ISP) for 2004-2008, USAID developed Strategic Objective (SO) 13: Capacity to Anticipate and Manage through Shocks Increased. This SO assigns disaster management responsibilities within the USAID mission cross-sectorally. It also commits USAID to working together with GFDRE “to incorporate access in a livelihoods framework” and to support the government to develop “nationwide livelihood baselines against which the impact of shocks will be measured.” Support to the DPPA in establishing the LIU is clearly in line with the purpose of SO13. It will also complement other aspects of support associated with SO13, and included in the Emergency Preparedness Support Program (EPSP), which since 2003 has been providing training and technical assistance to the DPPA to enhance its capacity building efforts. The USAID-funded Famine Early Warning Network (FEWS-NET) project has been implemented since 2000 by Chemonics with F.E.G. as the subcontractor responsible for directing livelihoods monitoring activities. The project provides essential support to the GFDRE through livelihoods-based monthly food security reports using existing monitoring of climatic and socio-economic indicators, and livelihood information. FEWS-NET was contracted to provide technical assistance to a pilot livelihoods-based study in Southern Nations, Nationalities and Peoples’ Region (SNNPR) over an approximately twelve-month period, from late January to December 2005. This Task involved training federal and regional officials on the household economy baseline methodology, the implementation of assessments in each of the region’s 40 “livelihood zones”, the transformation of this data into almost 100 woreda-level reports, as well as the development and implementation of a methodology to use this baseline information in DPPA’s emergency needs assessment process. The SNNPR pilot livelihoods baseline implementation, supported through USAID’s EPSP, provided an important step in establishing a DPPA-led livelihoods-based needs assessment approach. This effort was a follow on, and consolidation of, two previous pilot exercises in Amhara and Somali regions, both of which were intended to help the Early Warning Working Group reach consensus on a common analytical approach to carrying out needs assessments. Overview of the approach HEA Household Economy Approach (also known as the Food Economy Approach) has been in active and growing use throughout Africa and parts of Asia and the Middle East since the early 1990s. At its core, HEA aims to show how households in different areas live in typical years, and how they will be affected by any number of ‘shocks’, ranging from natural hazards (drought, flood, cyclones) to market hazards (changes in price, closure of markets due to war) to health hazards (HIV/AIDS, guinea worm). Its first direct application in the context of a national assessment and early warning system took place in 1994, when SC-UK in South Sudan set up the Food Economy Analysis Unit within WFP/Operation Lifeline Sudan (OLS). Its rapid and widening uptake since then is due in large part to the fact that it is relatively simple, generates effective results, is applicable in a wide variety of contexts, and is transparent. In recent years, F.E.G. has spearheaded the technical evolution of the approach, developing a wide range of tools for different settings, and establishing training modules and procedures for a range of users. The conceptual approach rests on three basic steps: 1. livelihoods baseline development; 2. key parameters monitoring; and 3. scenario analysis. Because the livelihoods baseline incorporates B-2 information on both food and non-food arenas (organizing this data into wealth-group specific food options, cash income options, and expenditure patterns), it is possible to monitor and assess both food and non-food needs. For example, non-food monitoring and needs assessment has been carried out successfully in urban areas (Harare, Djibouti) using this approach. The three main steps in HEA are introduced below. Livelihoods Baseline: The livelihoods baseline consists of information sets about households which are generated through primary field work. The household is taken as the reference point because it is the chief unit through which populations operate for production, sharing of income and consumption. These sets of information are stratified by geography (livelihood zones), economic/status considerations (wealth or access groups), and finally quantified in terms of food, income and expenditure. The result is a database rich in detail, and useful for a wide range of analyses, including, but not limited to: household food deficit projections to assist in early planning, targeting, and response; cash and non-food in-kind needs determinations; poverty analysis; development planning; policy assessment and planning; and market impact studies. Key Parameters Monitoring: Key parameters are the set of monitoring variables (such as crop production, food prices, livestock prices, rainfall data, and changes in HIV prevalence and distribution.) associated with the most significant food, income and expenditure options in a particular livelihood zone. They form the basis for a livelihood zone-specific monitoring system. In particular, they help guide what monitoring information should be collected and when. Key parameters are necessary in order to establish the context for an early warning system. These key parameters are then tracked, and changes in them are assessed in relation to the livelihoods baseline. Scenario Analysis: Integrating livelihoods into the current early warning system in Ethiopia is essentially about introducing livelihoods baselines as analytical reference points during the main assessments. This is done through scenario analysis. Scenario analysis is the process of using livelihoods baselines in conjunction with key parameters data to establish the range of appropriate response options in both food and non-food terms. In Ethiopia, two major assessments take place, one in November/December and the other in June/July, corresponding to the end of the meher and belg harvests and pastoral-area rainy seasons, respectively. It is at these times that projections of annual needs are made and revised, forming the basis of international appeals and GFDRE/donor/UN/NGO response planning. The establishment of the Livelihoods Integration Unit While considerable progress was made in 2005 through the training of technical expert “core teams” within the federal EWD and SNNPR/DPPB office, it also became clear that until a larger critical mass of expertise had been trained in the HEA approach, there would be a need for continued consultant support and backstopping. For example, it is expected that trained staff will turnover, requiring the training of new personnel to take their place. Computer models based on key parameters will need to be developed, tested and fine-tuned. Further development of methodologies to apply the food economy approach to assessing non-food household needs is required. Pastoral and cropping areas are different, and require compatible but different methodologies. Different agencies working in various regions of the country using the same techniques will need to be coordinated. For these various reasons, until the livelihoods-based need assessment is stabilized, continued technical assistance within the National Early Warning System is required. Therefore, it was recognized by USAID/DPPA that there was a need to establish a “Livelihoods Integration Unit” within the federal DPPA Early Warning Department. Thus, as a continuation of the SNNPR work - USAID is supporting the establishment of a Livelihoods Integration Unit (LIU), within the national Early Warning Department of the DPPA. This unit will be committed to the goal of capacity building, with an objective of handing over its core functions to the GFDRE within three years. The LIU will achieve this through: ongoing training; developing training materials; establishing a nation-wide comparable set of regional livelihoods baselines; integrating these baselines into the existing early warning system; and gaining general B-3 agreement around national standards for livelihoods-based early warning and emergency food and non-food assessment. The LIU will employ specific strategies to ensure that its core functions are transferred to the DPPA/EWD within three years, including maximizing participation of the DPPA in project activities, standardization of approach, and information sharing and exchange of ideas between government staff at all levels. Every aspect of the LIU will be designed and implemented with an emphasis on capacity development. Building and maintaining partnerships will be a crucial determinant of the LIU’s success. Therefore, the approach we take will aim to maximize communication, collaboration and coordination with the DPPA/EWD and other key stakeholders. At the outset of the project, it was proposed that an evaluation would be conducted to review the pilot work done in SNNPR, Somali and Amhara regions. Subject to the recommendations of the evaluation, the analytical approach to be used in carrying out the remaining livelihoods baselines in Ethiopia will be consistent with the methods and procedures used in the SNNPR pilot exercise, as well as in the pastoral area work that has been, and continues to be, carried out by SC-UK with support from F.E.G. Whilst much progress has been made already in the area of livelihoods integration in the Early Warning System, there is need for further work in the areas of longer-term capacity building; better coordination among all stakeholders; refinement of non-food needs assessment methodologies; and development of appropriate analytical software models. F.E.G. will continue to be responsive to the requirements of the GFDRE and other stakeholders in the process of furthering these goals. USAID/Ethiopia indicated to the DPPA that an expansion of the livelihoods-based needs assessment system would be welcomed in other settled agricultural regions if the SNNPR pilot was successful according to several success criteria. Success criteria highlighted in the contract for the SNNPR pilot included: timely completion of reports; livelihoods-based emergency needs assessment methodology considered workable; data and results considered representative of SNNPR; changed attitudes of government staff and improved emergency programming; involvement of other line agencies; and official use of the livelihoods data in the 2006 Humanitarian Appeal. Following delays in the awarding of this contract – the initial evaluation has been delayed – however it is still anticipated that the results of this evaluation will feed into the final design of the program. The evaluation will be done by an international consultant supported by a representative from the DPPA. Objectives of the evaluation The objective of the evaluation is to review the successes of the pilot and to identify areas which require further strengthening and focus in the future; provide guidance on the expansion of the livelihoods-based needs assessment system to other regions within Ethiopia. (i.e. enable the LIU/DPPA team to incorporate the lessons learnt from the evaluation of the regional work into the design and development of the project). The evaluation should help highlight aspects of the pilot exercises where standards (eg specific national or SPHERE) are especially important. This evaluation will initiate an iterative process of monitoring and evaluation that will be built in to the LIU’s management system in order to incorporate what works and discard what fails. Activities The level of effort for this activity will be 24 working days to be performed in Ethiopia. The evaluator will be supported by DPPA staff pilot phase there. The evaluator do the following main tasks: • A rapid desk review of available literature on early warning systems in the Horn, and on the reports which emanated from the Ethiopian pilot exercises (in SNNPR, Amhara, Somali) • At federal level - consultations with USAID , DPPA, LIU and other Steering Group members (SC-UK, FEWS-NET) . B-4 • At regional level (this will focus on the SNNPR pilot) - interviews with team leaders from the pilot exercises, team members, government staff at the regional, zonal and woreda level The evaluation should review the development of the baselines including usage of the baselines within the SNNPR – benefits of the approach should be highlighted together with lessons learnt and implications for future programming. The evaluation findings/recommendations should be structured to feed into the LIU project objectives (and KRAs) listed below. Objective 2: National and Regional Staff Training (KRA 2) • Review of training approach (to include interviews with trainees) • Review the quality of implementation (both training and data collection, including documentation and training materials). • How has the capacity at federal and regional level to do livelihoods needs assessment and analysis changed as a result of the pilot? Objective 3: Early Warning System Integration (KRA 1.2) Objective 3b: Conduct baseline livelihood assessments (KRA 1.2) • Review the approach used including data collection and analysis tools used in the zoning, baseline development and scenario analysis. • Review the planning and implementation process of the pilot exercise. • Does the method, and outcome, provide information on the key questions decision-makers need answered in order to mount rapid emergency response. These questions include: who, where, when, for how long, and how much of what type of aid is needed in any given emergency. • How does this approach compare in terms of time and resources (with the previous EPSP methodology used in SERA) and to others used in Ethiopia Objective 3a: Integrate livelihoods based needs assessment into regular monitoring system (KRA 1.2) • Advise on the extent to which the baseline was linked to the monitoring during the seasonal assessments – how could this be strengthened? • Review application of the method in the Belg and Meher and identify the strengths and shortcomings and suggestions to address the shortcomings, including review of the 2006 Appeal to discern use of information Objective 4: Non-food Needs Assessment Methodology (KRA 3.1) • What data is being collected that could contribute to the assessment of non-foods needs beyond the expenditure deficit analysis carried out in SNNPR? • Review additional potential uses of the information for non-food and other purposes ( food security, development etc) Objective 5: National Livelihood Assessment Methodology Coordination (KRA 3.2) • Consult users particularly at the regional level (DPPB/FSCB, WFP and NGOs) to assess their understanding of the value of information and how it could be used – but where appropriate also at the national level (eg within USAID, and other organizations presently familiar with the work). This should include both how the data is being used and how they envisage that the information could be used in the future. The evaluation should place particular emphasis on looking at the success criteria identified by USAID – as outlined in the bid document: • timely completion of reports; • livelihoods-based emergency needs assessment methodology considered workable; data and results considered representative of SNNPR; • changed attitudes of government staff • improved emergency programming; B-5 • involvement of other line agencies; and • official use of the livelihoods data in the 2006 Humanitarian Appeal”. Deliverables. The evaluation team leader will submit the following deliverables for the completion of this scope of work: • Presentation in Ethiopia to USAID, LIU, DPPA, and the LIU Steering Committee of the recommendations and findings of the evaluation. • A final report detailing the findings and recommendations of the evaluation. This report will be structured to be incorporated into the final LIU design document. Both hard and soft copies of the report should be submitted. The main body of the report should be no than 40 pages in length. The final report shall include an executive summary that is to be no longer than 5 pages. Reporting and Relationships During the period of this assignment, the evaluation team leader will receive guidance from the USAID Assets and Livelihoods Transition Office Cognizant Technical Officer (CTO) Tigist Yifru and Chief Suzanne Poland. The team leader will report directly to the LIU COP Jane MacAskill. Annex A: Suggested reading list. Livelihoods Study to Support Improvements in the DPPA Emergency Needs Assessment Methodology. Terms of References and Organizational Structure. January 2005. SNNPR Steering Committee Guide to Rapid Food Needs Assessment. March 2004. FEG Household Economy Outcome Analysis. A guide for the Food Security Assessment Unit (FSAU), Somalia March, 2006 The Food Economy Approach: A framework for understanding rural livelihoods. 1998. RRN Tanya Boudreau Food Economy Group guide to Rural Livelihood Zoning. January 2006 Sample of presentations Samples of woreda and livelihood zone reports as well as regional overview Examples of spreadsheets and integrated spreadsheet Report on technical review of phase 1 of project. Improving food security monitoring and early warning capacity within the regional disaster prevention and preparedness bureau – Somali Regional state. SC-UK, 2003 Evaluation of the food security monitoring and early warning project – Somali region (May 2001 – November 2004). Tedla Desta consultant. SC-UK Chemonics International. Tasks Order Proposal for Livelihoods Baseline Study to Support Improvements in DPPA’s Emergency Needs Assessment Methodology. Washington, DC, November 3, 2004. Draft Scope of Work for Project Extension, Livelihoods Baseline Study to Support Improvements in DPPA Emergency Needs Assessment Methodology, SNNPR, Ethiopia. Addis Ababa, not dated. SNNPR Livelihoods Baseline Study, Steering Committee Meeting. Awassa, Ethiopia, June 16, 2005. B-6 Kumar-Range, Shubh, and Teferi Bekele. Evaluation Report for the Strengthening Emergency Response Capability (SERA) Project in Ethiopia. Addis Ababa, November 12, 2001. Lautz, Sue, and Yacob Aklilu, Erin Boyd. Assessments & Appeals: Strengthening Non-food Emergency Response in Ethiopia. Addis Ababa, September 23, 2005. Deliverables from the livelihoods pilot projects in SNNPR, Amhara, Somali regions – including presentations. Evaluation of the EPSP. FSAU/FEWSNET Somalia Reports. FEWSNET Djibouti Reports. www.fews.net Risk and vulnerability in Ethiopia learning from the past, responding to the present, preparing for the future. A report for USAID. Sue Lautze, Yacob Akilu et al 2003 SNNPR Feedback Workshop – Tuesday September 14, 2006. C-1 ANNEX C: MONITORING AND DATABASES. – DECEMBER 06 The need for further clarity on monitoring and data storage/analysis was identified by the LIU and other interested parties – as coverage of these topics in the draft evaluation was felt to be inadequate. This led to further discussions with interested parties – see section ii) below. This paper summarizes the methods of data storage used in the SNNPR pilot and stakeholders views on the monitoring and databases. i) Methods of data storage used in the SNNPR pilot. Box 1 below summarizes the various tools and data storage methods that were used in the SNNPR pilot (The box also includes notes on monitoring – which was not part of the SNNPR pilot). Databases used in the SNNPR were • Excel baseline spreadsheet used to store and further cross-check the baseline data, • Arcview used to develop livelihood zone maps included in the baseline profiles, • Excel integrated spreadsheet used to link the baseline data with key parameter information collected during the seasonal assessments. Box 1: Overview of tools/databases used in the pilot SNNPR project and presently in the LIU. Description of tools/databases Advantages Improvements planned using present tools Alternatives proposed by other stakeholders Comments Mapping Available data (shape files) is in Arcview. Presently there are 40 shape files for the 40-livelihood zones SNNPR region. Baseline data or monitoring data is not yet linked to the mapping information. Arcview common tool used for mapping. Can produce high quality maps and be used for basic GIS functions The development of one shape file all 40 LZs for the region would facilitate updating and usage. Use maps as a tool to present data from the baseline and enable visual comparisons between LZ and woredas to be made. This could be done by exporting Arcview data to excel, adding in baseline info and importing back into Arcview To link Arcview maps to an Access based database together with key parameter monitoring data. The development of a simple Access based database to incorporate population and baseline data. This should be able to import data from excel. An enterprise database was considered to be too expensive. An access based database would take time to develop and would require additional funding – so in the short term cannot replace the present the excel based database – but need not be ruled out in the mid term. However monitoring requirements need to be clarified (see below), prior to investing in an alternative database Single baseline spreadsheet These store all the data collected during the baseline assessment from community and wealth group interviews. The use of the spreadsheet enables the teams to view all the Most people have seen/used excel before (i.e. data is in a format that most people are able to use). Enables teams to input data in the field. Development of hyperlinks to help people move around the spreadsheet Training of baseline team leaders to strengthen the preliminary analysis at the field None Analysis using the spreadsheet gives teams a good understanding of rural economies. Contributes to discussions on appropriate thresholds that C-2 data they have collected. The data is then crosschecked and analyzed by the teams who collected the information. Simple to use for analysis and crosschecking of data. Teams can hide/unhide (columns/rows) data for analysis purposes. Facilitates analysis and ensures that the analysis is transparent – and not a black box. level. Structure the baseline analysis workshop to include additional training on the logic behind crosschecks. consider livelihoods rather than only food needs eg: • What should minimum non-food expenditure include? • What items should be considered as essential expenditure for households (health, education, inputs?) • Should the threshold for essential expenditure be the same for all wealth groups? Integrated spreadsheet and the seasonal assessment The IS enables teams to do a regional analysis of need. Supports the government policy of decentralizing Field teams can input their own data. Teams can see all the data from the neighboring woredas which provides additional crosschecks Removes politics from the discussion of needs. Introduces people to a livelihoods approach to looking at needs Continue to develop the IS to make it easier to use. Provide training on the use of the IS to both senior managers and field teams (including underlying concepts) Ensure that teams of people: DPPA, DPPB, WFP, UNOCHA and NGOs) can lead the analysis during seasonal assessments. None (see notes below on monitoring system) Highlights the impact of poor crop production on different wealth groups and also indicates the importance of labor rates and maize purchase prices particularly for poor households. Strengthens peoples knowledge of rural economies and excel. Monitoring A monitoring system that links up with the baseline information has not yet been finalized. Preliminary piloting is being initiated in SNNPR Priorities are something that is practical – that would enable woreda level staff to do initial monitoring. A simple system – that is sustainable given staffing constraints (eg high turnover) A system that enable federal level to have an overview of the situation within the country. For details of preliminary piloting – see summary below of meeting on 23 Nov, 2006. Monitoring requirements have not yet been clarified but does need: – To identify appropriate indicators that will link to the baseline data. – A monitoring framework that has predictive capacity. C-3 ii) Summary of key points from discussion on monitoring and databases with key stakeholders Whilst the notes below do not attempt to fully address the type of monitoring and data storage/analysis system that will be used in the LIU, they do attempt to bring some clarity to the debate and move the debate forward – in the hopes that over the coming months practical and sustainable approaches can be developed for both monitoring and the development of an appropriate database. Key points taken from a discussion on monitoring and databases - 23 November, 2006 Participants: FEWSNET: Alemu, Nigist, SC-UK: Esayas, Demeke, DPPA: Getachew, Abiy, Zenit, USAID: Tedi, LIU: Jane, Mark, • The present tools: the single zone (outcome analysis spreadsheet) and the integrated spreadsheet (IS) are both very nice and transparent (and considered to be appropriate for their present purpose). It is easy to see what is going on, there is no black box. It is manageable at both the woreda and zonal levels as number of analysis to be completed is very small. • The Integrated Spreadsheet was felt not to be appropriate for a federal level as opposed to a regional analysis. But the group differed over the importance given to this. Some felt that the emphasis should remain on developing a regionalized analysis in line with government policy, others felt that this would take time and a national system should be developed. • There is considerable interest among many of the participants in the development of a new database to linking monitoring and HEA data. • However the framework for analysis of monitoring and HEA data has not yet been developed to give shape to a database. • There is a considerable amount of data already available in both the baselines and from the seasonal assessments that could contribute to predicting outcomes, which is not presently fully utilized. There is a need to look at ways that this data could be more fully utilized for scenario analysis. • A preliminary framework for a simple monitoring system was outlined. This could initially be developed through developing 3 scenarios (worst, middle and best case scenario) during the seasonal assessment. Key parameters within these scenarios eg prices could be monitored over time to see which scenario is most likely to occur. • In the meantime there could continue to be discussions on/ a review of the present monitoring system and key parameters to look at how the simple monitoring system could be developed. • FEG & FEWSNET should continue to liaise on potential database development including mapping options. • To go forward and develop a database – it is essential that the purpose of any proposed new database is clearly defined (i.e. what is it that is needed from a database). Whilst the LIU budget limits potential development of a database at the present time. Interested parties could seek other sources of funding - if the purpose/added advantage of the database is clearly defined. • The present LIU budget is sufficient to fund a database position for somebody to manage/update the present excel based database, with limited technical support to improve this person. There are no funds within the LIU budget to develop new database. C-4 Possible framework for a simple monitoring system – outlined in the above meeting Ways of linking the monitoring with the HEA analysis have not been fully explored. One simple possibility would be to consider the consumption year - eg in cropping areas consider the production season – monitoring of cropping, harvest, seasonal assessment, hunger season, etc… and think in terms of 2 types of monitoring. The key parameters in each type of monitoring would be: I) Monitoring of problem variables – crop progress, prices, livestock, labor, etc. These would be relevant at different times of year – more so approaching seasonal assessment. Information on these parameters can be linked to the baseline and used to predict outcome. A simple way of monitoring potential outcomes would be during the seasonal assessment to prepare 3 scenarios, a worst case, middle and best-case scenario. These scenarios would focus on parameters that are presently being monitored eg prices – which could then be monitored over time. Monitoring would indicate which of these scenarios was most credible – and responses could be designed accordingly. II) Monitoring of outcome variable (eg food deficit, malnutrition). Nutrition status is the most important variable, others include: labor migration, excess livestock sales etc. These are outcomes, monitoring of these will be particularly important during hunger season. Stage I above would be the best attempt at prediction of potential outcomes and their impact on wealth groups and livelihood zones Stage II would provide a check – i.e. were the predictions effective/done properly. If stage 2 is not considered then the ability to predict/monitor will not improve over time.