Cairo Air Improvement Project Air Quality Monitoring Component Performance Evaluation for the Air Quality Forecasting Program for Greater Cairo Chemonics International, Inc. USAID/Egypt, Office of Environment USAID Contract No. 263-C-00-97-00090-00 July 2001 EXECUTIVE SUMMARY This document provides a review and evaluation of the first season of operation of the air quality forecasting program for Greater Cairo. The program has become known as the Early Warning System, and was developed in response to growing concern on the part of the Minister of Environment and EEAA regarding the frequency and severity of air pollution episodes during the Fall and Winter seasons. Cooperation of the Egyptian Meteorological Authority has been instrumental in ensuring that essential meteorological technical inputs and staff are available to the forecasting effort. USAID, through CAIP, has supported development of the initial particulate matter forecasting methods and related features. The review divides the program into two interrelated aspects, the forecasting system, including the technical tools and methods used to generate a forecast, and the overall forecasting program, encompassing resources, institutional arrangements, communications, and other aspects. The evaluation found that: 1. Available data suggest two general types of PM episodes (excluding Khamasin): northerly wind episodes dominated by agricultural burning and less sensitive to meteorological factors, and southerly wind episodes which are more sensitive to meteorology. More analysis of the drivers of these episodes is required and additional features will emerge. North wind episodes are extremely difficult to forecast due to the unpredictability of burning. 2. The statistical model, the primary component of the forecasting system during the pilot phase, failed to predict episode peaks but can be useful in predicting trends (except when high concentrations result from agricultural burning). 3. Additional forecasting techniques under development by EMA meteorologists show promise, although they are also not capable of predicting peak concentrations accurately. The meteorological data and forecast are able to provide the meteorologist with enough accurate trend information to warn of the possibility of Fall episodes, should burning take place to the north. They are also able to predict episodes evolving due to pollution sources within Greater Cairo, especially with southern and south western winds. 4. Forecast production, as designed, relied heavily on the output concentration from the statistical model. Greater subjective input from senior meteorologists and daily forecast tracking may have led to greater forecasting accuracy. 5. The staffing of the pilot forecasting center was outstanding and computer equipment adequate for the purposes of the statistical model. Long-term human and equipment resources have not yet been determined. However, a plan is now under development. Detailed recommendations are provided at the end of the report and include priority activities that will enhance the quality and accuracy of forecasts in the Fall of 2001 as well as other activities recommended for the remainder of 2001 and over the longer term. The priority recommendations highlight the following: • Forecasting center operations and logistics ™ Add and train other meteorologists for the forecasting center and put in place permanent computer equipment to replace computers on loan from other activities; develop arrangements to include EMA as lead agency for producing the forecast. 2 ™ Work out arrangements for real-time access to EIMP air monitoring data to enhance forecasting and allow tracking of changes. ™ Transmit relevant data and forecasts from EMA to ensure EEAA forecasting center is consistent with other sources. • Forecast methods development ™ Conduct technical analyses to enhance additional methods, including the ventilation index and the dynamic meteorological model; pollutant source emissions estimates are critical for enhancing the dynamic model but full emissions inventories are not required. ™ Develop better understanding of the role of agricultural burning in episode levels and develop alternative approaches to help plan for episodes driven by agricultural burning • Issuing and communicating the forecast ™ Eliminate single concentration value from forecast and produce simpler, descriptive forecast. ™ Issue guidelines to ensure that multiple methods are used to produce a forecast, including the subjective judgement of meteorologists. ™ Develop procedures to track the forecast and changing meteorology and air quality throughout the day (on predicted episode days as well as non-episode days). ™ Enhance forecast communications, including fostering understanding of episode causes and forecast uncertainty; formalize communications protocols and "chain of communication". ™ Expand EEAA staff available for forecast communications, including tracking and communicating changes throughout the day and providing staff back up for 7-day per week, 24-hour operations • Forecast verification and continuous improvement ™ Develop Standard Operating Procedures for critical aspects of the forecasting program. ™ Establish and follow procedures for routine forecast verification and analysis of "missed" forecasts. A conceptual diagram of a typical forecasting system is presented Figure ES-1. The figure shows the importance of 1) using several different techniques to produce a forecast and 2) relying on a combination of objective methods and subjective evaluation and judgement. The pilot system initially relied on one objective technique (statistical model) but has already progressed to include additional methods and the expertise of EMA meteorologists. The recommendations of this report, if implemented, should lead to a comprehensive system, similar to that in the diagram, and capable of supporting EEAA's long-term needs. 3 Air Quality Data Meteorological Observations and Forecasts Forecasting Method A Forecasting Method B Forecasting Method D Forecasting Method C Forecasting Method E Forecast A Forecast B Forecast C Forecast D Forecast E Inputs (objective) Method Computation (objective) Evaluation (subjective) Final Forecast Judgment (subjective) • Evaluation each forecast • Compare toclimatology • Consistency with experience • Judgement Final Forecast • Review of data quality Review Inputs (subjective) Figure ES-1. Schematic showing the process of generating a PM forecast. The pilot system relied almost exclusively on a single objective method (e.g., Forecasting Method A) with progressively more subjective evaluation and judgement. 4 1. INTRODUCTION 1.1 Overview of the Program The air quality forecasting program (referred to as the Early Warning System [EWS]) for Greater Cairo was put in operation in September 2000 in response to concerns about the increased frequency and severity of pollution episodes during the Fall of 1999. The GOE requested support from USAID, which tasked CAIP with supporting initial program development. In order to have a program operational in a very short time from approval of the work plan (less than four months) a team with staff from CAIP, EEAA and the Egyptian Meteorological Authority (EMA) was created to work on different elements: • the forecasting system—technical methods and expertise to produce a forecast—created by CAIP specialists from Sonoma Technology Inc. working with senior staff from EMA • the overall forecasting program—including forecast communication and forecast follow-up activities, as well as other operational elements such as staffing and resources, interagency cooperation, etc.—under the management and direction of EEAA with CAIP support as needed. This forecasting program supports broader EEAA and GOE activities in air pollution episode management and mitigation. Although Egypt, through EMA, has a long history of producing sophisticated meteorological analyses and forecasts, air quality forecasting represented a new activity. Interagency agreements, resource commitments, data use and transfer protocols, continuous operations, and forecasting and reporting methods all had to be developed. In addition, the historical record of appropriate air quality data was very small. Based on all of these factors, a Technical Working Group agreed in July 2000 that the initial forecasting system should: 1. forecast a single 24-hour average (midnight to midnight) particulate matter (PM10) concentration for downtown Cairo for the next day, with concentration trend indicators--up, down, or stable-- out to 48 and 72 hours 2. be based on simple, transparent methods and the available data (historical air quality and meteorological data) 3. constitute a pilot system to be evaluated after the first season of operation 4. be expandable and sustainable by Egyptian specialists The initial system was developed based on a statistical model that used linear regression and Classification of Regression Trees (CART) as forecasting methods. The model became operational in September 2000. The hope was to expand the system to include a variety of methods and the growing experience of the forecasting team—this would be analogous to other systems elsewhere in the world. Two highly qualified senior meteorologists from EMA were instrumental in the development of the system and continue to oversee its operation and enhancement. 5 1.2 Scope of the Review The focus of this review was on the performance of the forecasting system put in place for the Fall 2000 season. Additional forecasting methods developed or under development by EMA meteorologists at the EEAA air quality forecasting center are included in the review. These methods include: • Ventilation index and wind direction forecasts that use meteorological model output to examine how air (and pollutants) will be mixed and transported throughout the region • A meteorological model adapted to simulate PM10 concentrations in Cairo by emitting particles and allowing the model to transport and disperse these particles The evaluation also reviews some of the critical elements of the forecasting program which impact the production, communication, monitoring, verification, and improvement of forecasts. These include: • short-and long-term staffing and other resources • forecast reporting and communication techniques, including communication of forecast uncertainties and treatment of missed forecasts • lines of communication • air quality and meteorology data availability • ability to monitor changes in air quality and meteorology on a continuous basis and communicate changes in the forecast • ability to deal with episodes caused by burning that cannot be forecasted by meteorology-based methods 1.3 Defining Episodes for Analysis EEAA has developed an interim approach to defining air pollution episodes. Several stages have been defined, the first being an internal "attention" level when the 24-hour average PM10 concentration reaches 350 µg/m3 . For this review, an episode was therefore defined as any day when the average of the daily concentrations at Qualaly, Fum al Kalig, and Abbassyia was at least 350 µg/m3 . Daily concentration for each site refers to the 24-hr average of hourly concentrations from 0000 to 2400 Local Time (LT). Between September 15, 2000 and March 31, 2001, eight days reached the threshold: Episode Date Observed 24-hr Average PM10 (µg/m3 ) 3-Nov-00 480 4-Nov-00 380 25-Nov-00 380 5-Dec-00 360 6-Dec-00 400 30-Dec-00 360 31-Jan-01 400 28-Feb-01 540 6 To examine the forecasting techniques and their performance more closely, we chose four periods that encompassed six of these eight episode days. Table 1-1 lists these four study periods. We added a “ramp-up” and “ramp down” day to each of the episode days listed in Table 1-1 to examine air quality and meteorological conditions before and after the episodes. Table 1-1. List of episodes selected for detailed analysis. Each episode includes a “ramp-up” and “ramp-down” day. Observed 24-hr Average PM10 (µg/m3 ) Date Study Period Fum al Kalig Abbassyia Qualaly Average 2-Nov-00 1 215 231 254 240 3-Nov-00 558 427 481 480 4-Nov-00 445 296 392 380 5-Nov-00 250 162 263 220 4-Dec-00 2 232 259 253 240 5-Dec-00 339 352 365 360 6-Dec-00 430 372 374 400 7-Dec-00 238 160 166 180 29-Jan-01 3 169 73 173 140 30-Jan-01 369 222 326 300 31-Jan-01 592 205 385 400 1-Feb-01 338 181 214 240 26-Feb-01 4 252 88 155 160 27-Feb-01 495 134 263 300 28-Feb-01 790 300 550 540 1-Mar-01 322 80 208 200 Based on the analysis of historical data performed during the summer of 2000 and from a review of observations this past year, two types of PM10 episodes occur in Cairo that are relevant to this forecasting program: • Agricultural burning combined with northerly winds that transport smoke from the Nile delta and/or other nearby farming regions southward into Cairo. These episodes tend to occur from October to December and are extremely difficult to predict without knowledge of when, where, and how much agricultural burning is occurring. • Light southerly winds (or winds with a southerly component) that allow the “base-load” emissions of PM10 in the greater Cairo region to reach episode levels. These events typically occur in the winter months (December to March). Other episodes occur during the Khamasin, when a low-pressure system creates strong winds that blow dust and sand, typically in the spring. Khamasin patterns are forecasted by EMA and are not part of the air quality forecasting program. Figure 1-1 shows the back trajectories for some episode days listed Table 1-1. Back trajectories show where the air came from 36 hours prior to the episodes. The first and second study periods, 7 November 2-5, 2000 and December 4-7, 2000, were selected to represent northerly wind conditions when agricultural burning likely enhanced the PM10 levels measured in Cairo. The third and fourth study periods, January 29 to February 1, 2001 and February 26 to March 1, 2001, were characterized by winds with a southerly component throughout the region. The meteorological conditions during the study days are complex and should be analyzed in greater detail. 10 CAIRO 8 6 4 2 0 22 20 18 16 14 108 612 4 2 0 22 10 8 6 4 2 0 22 20 10 8 6 4 2 0 22 20 18 16 14 10 8 6 4 2 0 22 20 18 29 29.5 30 30.5 31 31.5 32 32.5 33 33.5 34 34.5 29 29.5 30 30.5 31 31.5 32 32.5 33 10 8 6 4 2 0 22 20 18 16 14 12 10 8 6 4 2 0 22 20 18 16 14 12 10 8 6 November 03, 2000 November 04, 2000 December 05, 2000 December 06, 2000 January 31, 2001 February 28, 2001 Figure 1-1. 36-hour back trajectories arriving at Cairo at 1200 GMT (1400 LT) for selected episode days. The height of the trajectory is 300 m agl (above ground level). Numbers inside symbols represent the time in GMT. Back trajectories were computed using NOAA’s Air Resource Laboratory’s Global FNL data and Hysplit model. 2. FINDINGS 2.1 Statistical Model Performance Performance of the statistical model was evaluated by comparing predicted PM10 concentrations with observed concentrations. The focus of the performance evaluation is on next-day prediction (i.e., the forecast issued 24 hours in advance) since it is the most useful for planning communication and mitigation activities. As shown in Table 2-1, the performance of the statistical model was lower than expected. Predicted concentrations were often lower than observed concentrations. In addition, examination of time-series plots showed predictions tended to lag the episode day. As with any forecasting technique or program, various uncertainties in the inputs, methods, outputs, or usage can decrease the accuracy of the forecast. Determining and quantifying the uncertainties is the key to improving this forecasting method. Subsections 2.1.1 to 2.1.3 examine the various aspects of the statistical forecast model. 8 Table 2-1. Analysis of the impact of meteorological forecast uncertainty on PM10 forecasts. Observed PM10 average is for the Fum al Kalig, Abbassyia, and Qualaly monitoring sites. Forecasts for PM10 represent the next-day forecast (i.e., issued one day in advance). PM10 forecasts were computed using forecasted variables and using just observations to simulate forecast. Date Study Period Observed 24-hr PM10 (ug/m3) Forecasted PM10 using model data (ug/m3) Forecasted PM10 using observations (ug/m3) 3-Nov-00 1 480* 220 160 4-Nov-00 380* 280 240 5-Dec-00 2 360* 240 260 6-Dec-00 400* 300 240 31-Jan-01 3 400 220 160 28-Feb-01 4 540 320 400 * Likely dominated by agricultural burning. 2.1.1 Uncertainty in the Air Quality Inputs Air quality inputs for the model were 24-hr average PM10 concentrations at the Fum al Kalig, Abbassyia, and Qualaly monitoring sites. To evaluate these data, we examined the quality of the 24-hr averages and the hourly data used to create the average. Almost all 24-hr averaged PM10 data were sufficiently complete with 75% to 100% of the hourly measurements used in the average. We found only one case on December 4, 2000, at Fum al Kalig when 24-hr averaged PM10 was only comprised of 9 hourly data values. Although this sample is somewhat unrepresentative, we estimate that it would have only increased the predicted PM10 from 300 to 320 µg/m3 . Finding: The PM10 data from Fum al Kalig, Abbassyia, and Qualaly sites were of sufficient quality and completeness for forecasting. The air quality data inputs did not significantly contribute to uncertainty in the PM10 forecasts. 2.1.2 Uncertainty in the Meteorological Inputs Forecasts of meteorological surface and upper-air variables were used as input to the model. The statistical model was developed using historical weather variables measured specifically at Cairo Airport and Helwan. Operationally, meteorological forecasts from a grid model of these weather variables (at a grid point close to Cairo Airport and Helwan) were used as input to the model. Thus, any errors or biases in forecasted weather variables can produce uncertainty in the statistical forecasts of PM10 concentrations. For each study period listed in Table 2-1, we compared the forecasted weather variables with the actual observations. In general, the comparison was good. As expected, some differences did occur, which is typical when comparing grid point forecasted values to site observations. 9 To examine the potential effect of forecast errors in the meteorological variables on the statistical model output, we re-ran the statistical model using observations for just six of the episodes days. As shown in Table 2-1, on the November and December episode days only modest changes occurred when meteorological observations were used to simulate perfect forecasts. For the January 31 forecast, using meteorological observations caused a puzzling degradation in the PM10 forecast to occur. However, for the February 28 episode, using the meteorological observations in place of the forecasted values would have improved the statistical PM10 forecasting and would have correctly predicted the attention PM10 level. We believe that during episodes that were strongly controlled by meteorological conditions (namely, southerly wind events), the model was more sensitive to forecast errors. However, during periods when smoke from agricultural burning was the dominant factor in producing high PM10, the weather’s influence on pollutant concentrations had little effect. Thus, the statistical model was fairly insensitive to errors in the meteorological forecasts. Finding: The meteorological forecasts are accurate. During episodes controlled by weather conditions (not smoke), the statistical model will be more sensitive to uncertainties in the meteorological forecasts. 2.1.3 Evaluation of Statistical Trend Predictions A fundamental problem with the statistical model is its inability to predict peak concentrations (which are rare events). However, the statistical model would often correctly indicate whether PM10 concentrations were expected to increase or decrease. One component in the model was a trend prediction that was based solely on forecasted meteorological variables. This section evaluates the accuracy of these trend predictions. The linear regression equation in the statistical model was used to forecast PM10 trends by using forecasted weather variables as input. Table 2-2 shows the percent of the statistical trend forecasts that correctly predicted the day-to-day observed trend in PM10 concentrations. Figure 2-1 shows regression trend forecasts for a 13-day period. The day-to-day trend in forecasted PM10 concentration is the important factor, not the magnitude. Data in these figures suggest that: • During periods dominated by smoke from agricultural burning, the trend forecasts were not very useful for day-to-day changes in PM10 concentrations. In other words, the meteorological factors appear to have little influence in increasing or decreasing PM10 concentrations when overwhelmed by smoke. • Trend forecasts perform better in December and January when the contribution of smoke may not be as significant and meteorological conditions are the dominant factors in producing PM10 episodes. In fact, the correlation of the forecasted PM10 with the observed PM10 concentrations was reasonably good (R2 of 0.47 – typical R2 in established forecasting programs only reach 0.70), considering the simplicity of the regression equation used and the complexity of the problem. 10 Table 2-2. Performance of the regression equation at correctly predicting the trend in observed PM10 concentrations. Month Regression Correctly Forecasted Observed PM10 Trend October 53% November 38% December 70% January 60% February 37% Figure 2-1. Time series of the regression-derived “next-day” PM10 forecasts and the observed PM10 concentrations for January 23 to February 4, 2001. Finding: PM10 trend forecasts from the statistical model provided advanced warning of when PM10 concentrations would increase or decrease during episodes associated with southerly winds and no agricultural burning. 2.2 Other Forecasting Techniques 2.2.1 Ventilation Index A ventilation index is commonly used in fire weather forecasting and for planning agricultural burning to determine how quickly and effectively smoke or air pollution will be flushed from an area. The concentration of air pollution over a city is affected by how fast the wind moves air through the region and the extent to which pollutants are dispersed vertically. A ventilation index provides a good measure of the combined effects of these processes on PM10 concentrations. 11 A forecasted ventilation index tool was developed by the EMA senior meteorologists and used informally as part of the air quality forecasting system during part of Fall/Winter 2000. The ventilation index was calculated using output from the EMA's Eta forecast model, which is a numerical meteorological model that predicts weather variables (winds, temperature, humidity, etc.) over northern Africa, Europe, and western Asia (see http://nwp.gov.eg/nwp/egeta/html/egeta.html). To compute the forecasted ventilation index, hourly predictions of wind speed and planetary boundary layer height (PBL) were extracted from the model at the grid value closest to Cairo. In addition, hourly wind directions were also extracted from the model to assist in the interpretation and use of the ventilation index. The ventilation index was computed with the following equation: 10 PBL *WS VI = where VI = ventilation index PBL = planetary boundary layer height (m) WS = average wind speed within the PBL (m/s) The ventilation index and wind direction were used as criteria to anticipate periods of stagnation when PM10 concentrations might be high. The criteria are as follows: • If wind direction is northwest, north, or northeast, winds will typically be strong and persistent; there will be little or no stagnation and if there is no burning, low PM10 concentrations are likely. • If wind direction is south or southwest (180 to 240°) for more than six hours and if the ventilation index is less than 4 for more than six hours, then a period of stagnation and higher PM10 concentrations are likely. An example of the forecasted ventilation index and wind direction compared to the observed PM10 concentrations is shown in Figure 2-2. This figure illustrates how this forecasting method can be used: • Forecasts of southerly or southwesterly winds and a ventilation index less than 4 corresponded well with actual observed values of high PM10 concentrations (≥ 400 µg/m3 ). • Although the forecasted ventilation index was low (i.e., ≤4) for the first 30 hours, the forecast wind direction was not from the south/southwest; thus, the criteria were not met and actual PM10 concentrations were generally less than 200 µg/m3 . The ventilation index and wind direction provide a good qualitative forecasting method to anticipate when stagnation will likely occur, but they do not quantitatively predict PM10 concentrations. Finding: Using the ventilation index forecasting technique provides a good qualitative prediction for the south wind, “base load” emission episode types. 12 2.2.2 Simulating PM10 Concentrations with Dynamic Meteorological Modeling The Eta weather prediction model run twice daily by EMA provides forecasts out to 120 hours of winds, temperature, rain, etc. An extension of this model was adapted to simulate PM10 concentrations in Cairo. Work is in its early stages and the model has not yet been used as part of the operational forecasting system in Cairo. The focus of this section is to further examine the PM10 simulations using the meteorological model and to recommend future development activities. The model used for the PM10 simulations was the National Center for Environmental Prediction (NCEP) Sigma model, which is an extension of the NCEP/Eta model. It runs on (1) a regional scale, which provides initial and boundary conditions for the local-scale model, and (2) a local scale, which provides high resolution simulations of the weather in the Cairo and Nile delta areas. To simulate PM in the model, four illustrative emission point sources (in the absence of true emissions values) were added to the model. These sources, located in the vicinity of Tebbin, Helwan, Cairo Airport, and Shubra El-Khima, were continuously emitted into the model PM with concentrations of 1500, 1000, 100, and 100 µg/m3 , respectively. Note that these are relative estimates of emission from a few sources. As the model runs, the PM concentrations were advected horizontally and vertically, dispersed horizontally and vertically, and removed by wet and dry deposition. Graphical views (and animations) like those shown in Figure 2-3 allow forecasters to watch PM concentrations move and change throughout the model’s domain. In addition, hourly predictions of relative PM concentrations can be extracted from the model. Figure 2-4 shows the hourly predicted PM for Greater Cairo and averaged hourly observed PM10 concentration at the Fum al Kalig, Abbassyia, and Qualaly monitoring sites. The model predicts increasing and decreasing PM concentrations which were generally observed over this four-day period. A number of assumptions were involved in performing this initial modeling effort, and should be addressed in future work to improve the model performance and accuracy: • The simple input emissions, which only reflect part of the PM10 sources in Cairo, are assumed to represent all the sources. (Accounting for dust and mobile sources and properly distributing the emission sources throughout the model domain will help improve the forecasts). • The modeled PM only accounts for the primary particulate matter. CAIP's recent source attribution study indicated that 15% to as much as 50% of the PM10 results from secondary chemical reactions. Finding: Using the meteorological model to simulate PM10 transport and dispersion may prove to be very useful in improving the PM forecasting program. It provides another forecast tool and offers several long-term benefits, such as • Separating emissions from different sources (mobile, point, etc.) and examining the relative contribution of the sources at receptor locations. •Running several different emission scenarios, including model runs with and without agricultural burning, to look at the relative impact of smoke or PM10 in Cairo. 13 Figure 2-2. Data used for the ventilation forecasting technique. (a) Forecasts of hourly wind direction from the Eta model, (b) forecasts of ventilation index computed using output from the Eta model, and (c) hourly PM10 concentrations averaged from Fum al Kalig, Abbassyia, and Qualaly. The forecast was issued on January 29, 2001, at 00 GMT (02 LT) for the 72-hour period from January 29 to January 31, 2001. 0 50 100 150 200 250 300 350 0:00 5:00 10:00 15:00 20:00 1:00 6:00 11:00 16:00 21:00 2:00 7:00 12:00 17:00 22:00 Wind Direction (°) 0 2 4 6 8 10 12 14 0:00 5:00 10:00 15:00 20:00 1:00 6:00 11:00 16:00 21:00 2:00 7:00 12:00 17:00 22:00 Ventilation Index 0 100 200 300 400 500 600 700 800 900 29 -00 29 -05 29 -10 29 -15 29 -20 30 -01 30 -06 30 -11 30 -16 30 -21 31 -02 31 -07 31 -12 31 -17 31 -22 Day - Hour (LT) PM10 Concentration (ug/m3) a b c 14 Figure 2-3. Example output from the PM10 simulations for 00 GMT (02 LT) for (a) a 12-hour forecast for January 29, 2001, (b) a 36-hour forecast for January 30, 2001, and (c) a 56-hour forecast for February 1, 2001. Shading denotes relative PM10 concentrations, and wind barbs indicate surface winds from the meteorological model. Note: concentrations are illustrative and give relative trends but are not intended to be actual predictions. 15 Figure 2-4. Observed and model PM10 concentrations for Cairo (average of three stations). Forecast was made on February 26, 2000, at 1200 GMT (1400 LT) for February 26 through March 2, 2000. 2.3 Forecast Program Elements 2.3.1 Operations and Logistics The pilot forecasting program benefited greatly from close cooperation among different offices within EEAA and between EEAA and EMA. Clear agreements with EMA and cooperation of the EMA Chairman and senior staff meant that essential meteorological expertise, data, and model outputs were available in a timely fashion for forecasting system development and operation. Daily 0 100 200 300 400 500 600 700 800 900 1000 26 -12 26 -19 27 -02 27 -09 27 -16 27 -23 28 -06 28 -13 28 -20 01 -03 01 -10 01 -17 02 -00 02 -07 Observed PM 10 Concentration (ug/m3) 0 100 200 300 400 500 600 700 800 900 1000 26 -12 26 -19 27 -02 27 -09 27 -16 27 -23 28 -06 28 -13 28 -20 01 -03 01 -10 01 -17 02 -00 02 -07 Simulated PM10 Concentration (ug/m3) a b 16 transfer of air quality data from EEAA's EIMP monitoring network to the forecasting center was also key to successful operations. Staffing of the forecasting center by EMA meteorologists, accustomed to 24-hour, 7-day per week operations, greatly facilitates transfer of EMA model-derived data and timely forecasting. However, staff commitments for the future are uncertain as two senior meteorologists, important to other EMA activities, have been dedicated to the pilot phase. Internet connections from the EEAA forecasting center are essential to transferring EMA and EIMP data as well as viewing other EMA-based meteorological information. The connection functions adequately. However, processing power of the currently available computers in the forecasting center is limited. In addition, the computers are apparently committed for other purposes within EEAA over the long term. Transfer of EIMP air quality data to the forecasting center usually occurs properly but has two significant imitations for forecasting: 1. Data pass through two points (Cairo University and the EEAA EIMP unit) prior to being available to the forecasting center. Staffing at these two locations was never intended to support time-sensitive, 7-day forecasting operations. Holiday, weekend, and other backup coverage is not always optimal to support forecasting. 2. EIMP data is not routinely available at EEAA throughout the day to permit forecast tracking and revision, in addition to other analyses. Finding: Pilot phase operations worked quite well but longer-term operations require further agreement on staffing arrangements, dedicated computers, and enhanced EIMP air quality data transfer and availability. 2.3.2 Forecast Communication, Follow-up, and Verification As agreed by the Technical Working Group, the forecasting system was designed to produce a single forecasted concentration, representing an average 24-hour concentration for downtown Cairo. Particularly at the beginning of the operations phase, this single value was communicated to decision-makers in EEAA and the Ministry of Environment with little discussion of uncertainties or the potential for alternative outcomes. As the forecasting team gained more experience and understood the limitations of the single statistical method, they developed alternative methods and informal guidelines and factored their own professional judgement more heavily into forecast communications. Nonetheless, the final forecast from the EEAA forecasting center relied heavily on the single concentration produced by the statistical model. When it became clear the model by itself could not forecast peak concentrations, the overall forecasting program lost credibility. Except in episode situations, it appears there was not a mechanism for forecast tracking during the pilot period. The forecasting effort was generally a once-a-day activity, not a program with regular (e.g., hourly) review of information and verification of the forecasts. This resulted in part from two constraints: 17 1) lack of routine access to hourly EIMP air quality data throughout the day and night 2) a limited number senior staff at EEAA available to oversee forecast tracking and communications on a continuous basis, 7 days per week Finding: The daily forecast has evolved from relying exclusively on one forecasting method, to using additional objective methods and the subjective judgement of the forecasters. This should continue to develop. Follow-up, tracking, and further communications related to the once-daily forecasts does not happen on a routine basis due to limited resources available during the first season of operation. 3. RECOMMENDATIONS Table 3-1 on the following pages provides recommendations for the forecasting system and overall program. Recommendations are divided between those most critical for the Fall of 2001 and those potentially begun during the Fall but continuing over the longer term (e.g., next 12 months). The process of producing the forecast, as well as communicating it is critical to maximizing the success of anticipating episodes as well as preparing for conditions different from the forecast. The text box below summarizes a typical process for generating a forecast based on objective and subjective steps (see also Figure ES-1). Producing a Forecast Using Subjective and Objective Methods There are both objective and subjective aspects of generating a daily PM forecast. The objective tools include the statistical and physical models presented in Sections 2.1 and 2.2. These models require air quality and meteorological inputs. Although these models are considered objective, the forecaster must subjectively review the input data, checking for consistency and meteorological reasonableness. When the forecaster reviews the data and observes an input parameter that is not reasonable or consistent, the forecaster must replace the value with a more reasonable one. For example, if the wind speeds appear too high, given the synoptic weather condition, the forecaster should either lower the input value and/or convey the uncertainties in the final PM forecast due to the uncertainties in the wind speed. After generating the objective forecasts from each tool, the forecaster analyzes the outputs and ask questions such as • Are the objective forecasts similar? If not, why? • Are some or all of the forecasts consistent with the forecaster’s subjective opinions? • Which method has produced the best forecast under similar weather conditions in the past? • Which one did best yesterday? • What are the uncertainties associated with each forecast? For example, is the timing of a weather event, such as a cold front, critical to the forecast and is this event increasing the uncertainty in the forecast? After asking these kinds of questions, the forecaster must decide what PM concentration to forecast and what the uncertainties are that are associated with the forecast. For example, based on his/her knowledge, experience, and observations, the forecaster may decide that all of the objective forecasts are erroneously because of unaccounted-for smoke, and the forecast should be much higher for that day. On another day, the forecaster may decide that the objective tools are consistent with each other and with his/her subjective analysis; thus, the average objective forecast should be used on that day with high confidence. On still another day, the forecaster may not be sure what to forecast because of unusual emissions or weather pattern. In this case, the forecaster must communicate his/her uncertainties associated with the final PM forecast. In summary, for a forecast program to be successful, the forecaster must use judgment when issuing the final forecast. It is critical that the forecaster communicate the rational for, and all of the uncertainties associated with, the final forecast. This gives the users of the forecast the necessary information to make good decisions. 19 Table 3-1. Recommendations for Air Quality Forecasting System and Program Enhancements. Task and Subtasks Purpose Action Priority for Fall 2001 Other 2001 & Longer Term Operations and Logistics • Develop a way to automatically collect hourly PM10 data from EIMP monitoring sites and distribute them to the Forecasting Center on the 8th floor at EEAA. This process should be as reliable as possible, addressing staffing shortages on weekends and holidays. √ • Set up procedures to review the hourly PM10 data. These procedures will allow forecasters to monitor the real-time data and make short-range predictions of when and where PM10 concentrations will reach episode levels. Document the procedures in simple “nowcasting” guidelines. √ Faster access to real-time data Real-time data enables 1) tracking of changing conditions, 2) ability to modify forecast and make short-range “nowcasts” (1 to 6 hours), and 3) better understanding of chemical/atmospheric processes. . • Based on “nowcasting” guidelines’ performance, modify the guidelines. √ • EEAA to develop long-term arrangement to include EMA as the lead agency for air quality forecast production; add two meteorologists, to be supervised by current senior meteorologists. √ • Acquire two permanent computers for the forecasting center (to replace two on loan) including one high-end model to improve computational processing and allow for higher resolution simulations. The high-end computer should be a 1.5GHz dual processor with 512MB RAM to ensure fast computations. √ Staffing and equipping the forecasting program Adequate staff and resources are needed to achieve seven-day-a￾week, year-round forecasting. • Implement a staffing program to ensure 7-day, year-round coverage. √ • Set up a process so that all relevant weather and air quality forecasts are transmitted to the Forecasting Center on the 8th floor. The EWS forecasters would then be responsible for consolidating and synthesizing the information and issuing consistent air quality forecasts. √ Forecast consistency with other sources EEAA air quality forecasts should be consistent with sources such as EMA weather forecasts. • Have EMA fax its daily stability forecast to the Forecasting Center. This forecast will be used as one of several tools to arrive at the final air quality forecast. √ • Review the overall forecasting program and identify critical functions/components (e.g., data transfer methods, staff backup). √ Redundancy procedure development The critical functions of the overall forecasting program ensure continuous operations. • Develop and document backup/alternative procedures and review with forecasters. √ Forecast Methods Development • Re-derive the statistical equations using three years of data (1998 to 2000), excluding periods of agricultural burning. This will provide a forecasting tool to help predict the increasing and decreasing trends in PM10 concentrations and will likely improve performance of the entire forecasting program. √ • Select new predictor variables that use some of the forecasted variables from EMA’s Eta model. This will allow a greater selection of predictor variables not available when the statistical model was developed in 2000. √ Statistical model improvements More data for the statistical model will improve the ability to predict increasing and decreasing trends in PM concentrations. • Develop guidelines for using statistical trend forecasts to anticipate episodes. √ • Use high-resolution meteorological model data to compute the ventilation index to get better forecasts of the PBL height and winds in the Cairo area. Specifically, run the Eta model at a 5- to 10-km resolution and use the grid￾point data nearest downtown Cairo to compute the Ventilation Index. √ • Set up an operational process and guidelines to use this method each day in the overall forecasting program. This will require running the meteorological model each day and using its output to compute the Ventilation Index. √ • Verify Ventilation Index and wind direction criteria; compute forecast verification statistics. √ Ventilation Index improvements Developing another prediction tool will provide forecasters more information and help improve the accuracy of the PM10 forecasts. • Further develop and refine the forecasting criteria. √ • Develop a gridded file of relative emission source profiles that includes components of the primary PM10 from dust and mobile, stationary, and point sources, with diurnal and perhaps day-of-week variation. Also, develop relative emission source profiles for agricultural burning in the Delta. √ • Set up an operational process to run the model on a high-resolution local scale; begin using the PM predictions as another forecasting tool. √ • Begin a verification program that compares the observed data to the model predictions, and adjust and refine the model and/or emission inputs. √ Dynamic model improvements Developing another prediction tool will provide forecasters more information and help improve the accuracy of the PM10 forecasts. • Provide training to the forecasters regarding the air quality aspects of this modeling effort (particle modeling, PM chemistry, emissions estimates, etc.). √ 20 Task and Subtasks Purpose Action Priority for Fall 2001 Other 2001 & Longer Term Forecast Methods Development (continued) • Perform additional analysis of CAIP’s source apportionment results to estimate the smoke contribution to PM in Cairo. √ • Examine the PM10 data collected in the Nile Delta, Kaha, and Cairo for meteorological conditions associated with the transport of smoke into Cairo. √ • Acquire and examine high-resolution satellite images of the Nile Delta region during the agricultural burning season. Estimate the extent of the burning and transport of smoke. Understanding √ PM influenced by agricultural burning Knowing when agricultural burning occurs, and how it contributes to PM10 in Cairo, will help forecasters better anticipate high PM concentrations during the fall burning season. • Investigate using meteorological forecasts to plan and schedule agricultural burning activities. Examine burning programs in the United States and other counties that use forecasted meteorological and air quality conditions to advise when, and what amounts of agricultural refuse, to burn. √ Issuing and Communicating the Forecast • Eliminate single concentration value from forecast, considering simpler, descriptive forecast. √ • Develop guidelines to ensure multiple forecasting methods are used to generate forecast and final forecast includes discussion of forecast uncertainty and factors that could change the outcome (e.g., meteorology changes) √ Modify the forecast format to include lessons learned from pilot phase. • Develop operational procedures for tracking the forecast and changes (meteorology and air quality) throughout the day and communicating changes to EEAA decision-makers. √ • Assign additional EEAA staff member for communications back up and develop a “Chain of Communication” so that there is a clear path of who communicates the forecast to whom. √ • Develop a communications protocol so that the forecasters know when, where, and how to communicate the forecasts. √ • Communicate uncertainty in all forecasts both in paper report and the verbal discussion. √ Methods to improve forecast performance and overall communications Improving the methods of communicating information can often improve how the forecasts are used and perceived. • Educate communicators about the meteorological conditions that produce episodes as well as the difficulties of predicting PM10. A forecast and its uncertainty are more likely to be properly and accurately communicated by a well-informed communicator. √ Forecast Verification and Continuous Improvement • Conduct case studies of the conditions that produced each type of PM10 episode and distinguish the common features of each episode. √ • Examine the large-scale weather patterns that produce PM10 episodes in Cairo. √ • Examine the smaller-scale meteorological conditions that produce and transport PM10 in Cairo. √ Data analysis Analyzing data enables a better understanding of the meteorological and air quality phenomena and physical mechanisms that control PM10 concentrations in Cairo. • Develop a series of conceptual models (written descriptions) of the weather and air quality conditions that produce each type of episode. √ • Begin developing a list of daily, weekly, and monthly tasks needed to prepare, issue, and distribute forecasts. √ • Write a SOP document with these tasks and review with the forecasters. √ Development of Standard Operating Procedures (SOPs) SOPs help ensure that all tasks are completed and that greater consistency exists among forecasters. • Continue to develop and revise the SOP. √ • Set up procedures to review the forecast each day. √ • Develop spreadsheets to track forecast performance and computer forecast verification statistics such as accuracy, bias, false alarm rate, skill score, etc. √ Verification program development Quantify the performance of the forecast program and identify performance trends with time. This process leads to better forecasts. • If forecasts are significantly missed, perform a retrospective analysis to examine what occurred; include an analysis of the actual air quality and meteorological conditions and any modifications to the forecasting procedures. √