A COMPARISON OF MEX GUIDANCE VS

NAM-CMAQ Ozone Forecast Verification for the Summer of 2005 at Knoxville, Tennessee

By Loren C. Marz

1. Introduction and Background

In the summer of 2004, the National Weather Service (NWS) launched experimental ozone forecasts for the north-eastern quadrant of the Continental United States (CONUS). The domain of this experimental ozone forecast capability encompassed most of Tennessee. In the summer of 2005, this domain was expanded to include the entire eastern part of the CONUS through the Mississippi Valley. The portion of the domain that had been in development during 2004, and experimentally tested in 2005 was approved for operational deployment in August, 2005. Thus, all of Tennessee is now part of the operational domain.¹

The North American Mesoscale (NAM) and Community Multi-scale Air Quality (CMAQ) models form the backbone of the new NOAA EPA air quality forecast capability. NOAA

runs the linked model system to provide ozone forecast guidance data for both state and local air quality forecasters and the public. State and local air quality forecasters issue the official air quality forecasts for specific regions of their respective states. Approximately 300 cities nationwide issue air quality alerts based on ozone forecasts and about 100 of them include information on airborne particulate matter. State air quality forecasters have historically used statistical models to provide air quality forecasts. These statistical approaches relate forecasted temperatures and sky cover to forecasted ambient ozone levels.²

As described by Otte, et al.³, the CMAQ model is coupled with the 12 km resolution NAM model to provide air quality forecast guidance for ground-level ozone (GLO). The NAM model provides predicted meteorological parameters to the air quality modules PREMAQ and CMAQ, including temperature, winds, mixing heights, and cloud cover. The pre-processor called PREMAQ modifies estimated emissions based on EPA’s National Emission Inventory for weather dependence. Interface processors convert the NAM output to a form that can be used by the chemical transport model CMAQ. The CMAQ then performs atmospheric reactive chemical transport simulations which are used to predict ozone concentrations.⁴

GLO is commonly formed (ref. e.g., #5) when nitrogen dioxide (NO₂), which is one of several nitrogen oxide species collectively called NOx, photo-dissociates into nitric oxide (NO) and atomic oxygen (O). The resulting O combines with molecular oxygen (O₂) to form ozone (O₃). However, the NO that is left over from the initial photo-dissociation immediately reacts with ambient O₃ to reform NO₂ and O₂. Thus there is no net ozone production just from the photo-dissociation of NO₂.⁵ Volatile Organic Compounds (VOCs) play a significant role in GLO production in that atmospheric decomposition of VOCs produces peroxy radicals (RO₂^. and HO₂^.). These peroxy radicals oxidize NO (another component of NOx) into NO₂, resulting in more NO₂ for O₃ formation and less NO for O₃ depletion.⁶ Relatively high ambient levels of GLO are considered a human health hazard in that O₃ causes inflammation of lung tissues. The Environmental Protection Agency (EPA) has set a maximum National Ambient Air Quality Standard (NAAQS) of 0.08 parts per million (ppm) 8-hour average.⁷ A maximum 1-hour value of 0.12 ppm was used prior to implementation of the 8-hour average, and is still used for areas classified as Early Action Compact (EAC) areas.⁸

A study was conducted for the 2005 ozone season, which is nominally between May 1 and September 30, to assess performance of the NAM-CMAQ ozone forecast guidance. The city of Knoxville, Tennessee was used as the forecast point. The NAM-CMAQ ozone predictions for Knoxville, Tennessee were compared to monitoring data subsequently obtained for the corresponding days. Maximum one-hour and eight-hour average ozone forecast guidance concentrations were used. Ozone concentration values are given in parts per billion (ppb). Due to the photochemical nature of ozone production, diurnal in addition to seasonal cycles are typical. Diurnal maximum 1-hour and 8-hour ozone levels are generally seen during afternoon hours.

2. Procedure

Ozone forecast guidance for Knoxville, Tennessee was obtained each day through the study period from the NWS ozone forecast guidance data base on their operational web site.⁹ The grid point used for Knoxville, Tennessee corresponded to latitude 36.00 N and longitude 83.90 W. Generally, data from the 12Z run were used, although other model run times were used in isolated instances when 12Z data was not available or where circumstances prevented obtaining the 12Z data. The ozone forecast guidance was printed out and the highest predicted one-hour and eight-hour values were determined. These values were then compiled for each day through the study period and input into a matrix (Table 1). A graphical depiction of the maximum 8-hour predicted compared to actual ozone values is provided in Figure 1. Forecast guidance data was not recorded on a total of 16 days. An entry of “MM” was made for each of these missing values.

Figure 1. Graph of predicted and actual maximum 8-hour average ozone values through the study period.

Official ozone monitoring data was obtained from the State of Tennessee Department of Environment and Conservation (TDEC) and the Knox County Department of Air Quality Management. There are two monitors in the Knoxville area, one monitor located at 9315 Rutledge Pike, Mascot, Tennessee (36.01944 N, -83.87361 W), and the other located at 4625 Mildred Drive, Knoxville, Tennessee (36.084722 N, -83.764722 W). Monitoring values for the two sites were averaged together. The monitoring data was also input into the matrix on the corresponding days. The NAM-CMAQ ozone forecast verification was based on the results of these two monitors.

The predicted ozone values were then compared to the monitoring results. Mean Absolute Error (MAE) and Mean Bias Error (MBE) were subsequently calculated for each predicted value for both 1-hour and 8-hours.

3. Results

The MAE for the entire study period was approximately 10.3 ppb for both the 1-hour and 8-hour maximum average values. The maximum 1-hour ambient ozone levels averaged 66.9 ppb, and the maximum 8-hour ambient ozone levels averaged 57.6 ppb through the study period. Therefore the 10.3 ppb MAE equates to approximately a 15.4% and 17.9% forecast error, respectively.

The NAM-CMAQ forecast guidance for Knoxville showed a high bias, which suggests a tendency to over predict ozone levels. The MBE for the maximum 1-hour average values was a +2.91 ppb. The MBE for the maximum 8-hour average values was +5.91 ppb. This over prediction bias has been reported previously in statistical evaluations for the entire Eastern domain conducted in a previous study of the NAM-CMAQ during developmental testing in 2004.³

The NAM-CMAQ predicted that 10 days would exceed the maximum 8-hour NAAQS limit for ozone, which is 0.08 ppm or 84 ppb, through the study period. Only one day actually exceeded the 8-hour value, based on the average of the two Knoxville ozone monitors. The 8-hour value was exceeded on July 26 when the average concentration of the ozone monitors was 92 ppb. The NMA-CMAQ predicted a value of 96 ppb for that day. There were no days which exceeded the now defunct maximum 1-hour NAAQS value of 0.12 ppm or 124 ppb, and none were predicted by the NAM-CMAQ.

Some spatial variability was noted between the two ozone monitors. There were two instances through the study period where one of the monitors exceeded the 8-hour ozone NAAQS, while the other was below, and the average of the two monitors was at or below 84 ppb. These instances occurred on June 24, when the Mildred Drive monitor recorded 88 ppb, and on August 12, when the same monitor recorded 86 ppb.

The maximum 1-hour observed values ranged from 29 ppb on July 7 to 103 ppb on July 26. The maximum 8-hour observed values ranged from 18 ppb on September 26 to 92 ppb on July 26.

The maximum forecast error occurred on June 9 when the predicted ozone value was missed by 60 ppb for 1-hour and 46 ppb for 8-hours. The NAM-CMAQ predicted a maximum 1-hour value of 119 and a maximum 8-hour value of 94 ppb when the actual ambient ozone concentrations were 59 ppb and 48 ppb, respectively, based on the ozone monitoring data. These errors were caused by a bug in the interface processing between the newly updated global forecast system NWP model (GFS) that provides boundary and initial conditions to the NAM. The bug was diagnosed as a result of its impacts on predicted GLO, and the resulting crisis fix corrected the situation by mid-June.¹⁰

Another large high bias error was noted on August 14, when NAM-CMAQ missed the 8-hour ozone value by 40 ppb (91 ppb predicted, 51 ppb actual). The large error that resulted on this day was probably related to unusual conditions not predicted by the NAM model. A cursory review of the weather conditions reported at the Knoxville, Tennessee airport (TYS) reveals temperatures were near the climatology temperature for the date, but with fog reported until 1000 (LST), and broken to overcast sky conditions in all but one observation during the remaining daylight hours. The only observation that did not report a ceiling occurred at 1600 (LST), which is fairly late in the afternoon after the typical peak diurnal ozone production time. Based on raw NAM model data from that date, the predicted temperature (91^oF) was close to the actual temperature (89^o F). However, the predicted relative humidity values were generally below 70% at all levels (925 millibars to 300 millibars) through the daylight hours suggesting that clouds were likely not predicted through the peak ozone-generating daylight hours, thus leading to the relatively high ozone values predicted by NAM-CMAQ. At any rate, it is clear that weather conditions, particularly cloud cover, play a major role in ambient ozone production. Since ozone production is a photochemical process, any errors in the NAM predicted cloud cover would have a significant effect on the output of the CMAQ.

The largest low bias or under predicting occurred on August 20 for both the 1-hour and 8-hour categories. The NAM-CMAQ predicted 51 ppb maximum for 1-hour and 48 ppb maximum for 8-hours while the monitoring results showed 85 ppb and 71 ppb, respectively. This equates to an under prediction of 34 ppb for the 1-hour value and a 23 ppb for the 8-hour value. The local climate data for that date at TYS show few to scattered sky conditions through the daylight hours. The raw NAM data show quite high RH values (>85%) at 850 millibars throughout the daylight hours suggesting the possibility that NAM over predicted clouds at that level. This would cause CMAQ to significantly under predict the ozone levels.

The most accurate forecast guidance occurred on May 24 and July 15 when the NAM-CMAQ maximum 8-hour value was 50 ppb and 54 ppb, respectively, the same as the monitoring data. The 1-hour forecast guidance was only 1 ppb from the observed values.

A breakdown of the NAM-CMAQ forecast errors by percentage in various categories is provided in Table 2. NAM-CMAQ actually demonstrated a low bias for both 1-hour and 8-hour ozone levels early in the season (May). For the 1-hour and 8-hour ozone levels, there were 19 days and 16 days, respectively, where the NAM-CMAQ under predicted ozone levels during May. There were only 7 days and 9 days, respectively, that were over predicted. NAM-CMAQ tended to persistently over predict ozone levels during the months of June, July, August, and September. The months with the greatest number of days in which NAM-CMAQ over predicted ozone levels were July for 1-hour levels with 21 out of 28 days sampled (75%), and August for the 8-hour levels with 25 out of 29 days sampled (86.2%).

4. Conclusion

In general, NAM-CMAQ did a reasonably good job of predicting maximum ambient ozone levels for the following day and could be used to provide relatively good guidance for air quality forecasters. There was a tendency for NAM-CMAQ to over predict ozone levels. There were a few days when large errors in the ozone predictions were noted. These ozone prediction errors are most likely related to errors in the driving NAM weather prediction, and could potentially be mitigated by incorporating knowledge of the expected performance of the NAM.

Acknowledgement

The author wishes to thank the staff of the State of Tennessee Department of Environment and Conservation (TDEC) and the Knox County Department of Air Quality Management for providing the ozone monitoring data. Thanks also go to the staff of the National Weather Service Forecast Office at Morristown, Tennessee, especially David Hotz, Science and Operations Officer (SOO). Also, the staff of the National Weather Service Office of Science and Technology, especially Dr. Paula Davidson, National Weather Service Program Manager for Air Quality Forecasting, for helpful suggestions with the study and manuscript.

References

1. Davidson, P.M., National Air Quality Forecasting Capability: Progress and Plans, June 17, 2005, available at http://www.nws.noaa.gov/ost/air_quality/NAQFC_update_0605.pdf

2. Personal communications with State of Tennessee air quality forecasters.

3. Otte, T. L., et al.; Linking the Eta Model with the Community Multiscale Air Quality (CMAQ) Modeling System to Build a National Air Quality Forecasting System, Weather and Forecasting, 2005, Vol. 20, No. 3, pp. 367–384.

4. McQueen, J.; Lee, P.; Tsidulko, M; DiMego, G; Mather, R.; Otte, T.; Pleim, J.; Pouliot, G.; Kang, D.; Schere, K.; Gorline, J.; Schenk, M.; Davidson, P.; Update of the Eta-CMAQ forecast model run at NCEP operations and its performance for the summer 2004, In 3rd Annual Models-3 User's Conference, EPA, 18-20 October 2005, Chapel Hill, NC, Community Modeling and Analysis System, available at http://www.cmascenter.org/conference/2004/abstracts/Forecasting/mcqueen_abstract.pdf.

5. Cooper, C. D.; Alley, F. C.; Air Pollution Control; A Design Approach, Waveland Press, Inc., Third Edition, 2002, page 595.

6. Davidson, P.; National Air Quality Forecast Capability: First Steps Toward Implementation, May 7, 2003, Office of Science and Technology, National Weather Service, available at http://www.nws.noaa.gov/ost/air_quality/Davidson.pdf.

7. U.S. Environmental Protection Agency National Ambient Air Quality Standards (NAAQS). See http://www.epa.gov/air/criteria.html .

8. U.S. Environmental Protection Agency 1-Hour Ozone Information. See http://www.epa.gov/oar/oaqps/greenbk/oindex.html#List1 .

9. National Oceanic and Atmospheric Administration, National Weather Service, Air Quality Forecast Guidance for Knoxville, TN, available at http://weather.gov/aq/probe_aq_data.php?latitude=36&longitude=-83.9 .

10. Personal communications with the National Weather Service Program Manager for Air Quality Forecasting.

TABLE 1. NAM-CMAQ OZONE FORECAST VERIFICATION STUDY DATA

Summer 2005

Ozone Levels in PPB (Max 1-hour average/max 8-hour average)

Date CMAQ Forecast (1/8) Monitor. Results (1) Monitor. Results (8)

5/01 52/51 56 54

5/02 52/50 52 48

5/03 54/51 55 50

5/04 MM 58 54

5/05 51/44 44 37

5/06 55/51 63 57

5/07 68/62 82 77

5/08 67/62 75 71

5/09 65/62 81 71

5/10 64/57 50 41

5/11 64/55 68 61

5/12 84/75 93 81

5/13 72/67 90 78

5/14 MM 61 50

5/15 60/55 58 54

5/16 61/57 53 49

5/17 74/66 74 68

5/18 75/64 87 82

5/19 67/62 92 82

5/20 61/57 57 49

5/21 MM 56 51

5/22 52/46 66 49

5/23 MM 63 60

5/24 56/50 57 50

5/25 51/50 46 44

5/26 MM 60 54

5/27 66/59 75 67

5/28 45/44 60 54

5/29 66/58 50 41

5/30 45/37 59 55

5/31 60/58 68 63

Date CMAQ Forecast (1/8) Monitor. Results (1) Monitor. Results (8)

6/01 56/50 53 44

6/02 62/48 47 42

6/03 72/63 38 29

6/04 65/54 65 58

6/05 72/67 66 58

6/06 72/65 76 64

6/07 77/68 53 49

6/08 93/90 58 48

6/09 119/94 59 48

6/10 70/60 54 45

6/11 49/45 47 37

6/12 49/45 38 27

6/13 56/52 54 44

6/14 71/66 68 62

6/15 60/58 63 58

6/16 55/53 54 50

6/17 52/48 65 61

6/18 71/58 70 67

6/19 76/71 74 68

6/20 84/73 67 53

6/21 89/83 79 70

6/22 68/63 81 74

6/23 84/79 82 77

6/24 90/84 92 84

6/25 94/86 89 81

6/26 85/74 73 62

6/27 61/50 66 51

6/28 51/45 67 51

6/29 69/64 83 67

6/30 73/64 80 66

Date CMAQ Forecast (1/8) Monitor. Results (1) Monitor. Results (8)

7/01 70/60 99 70

7/02 62/55 65 58

7/03 60/59 52 37

7/04 75/69 74 60

7/05 58/54 54 46

7/06 79/74 37 33

7/07 47/35 29 25

7/08 61/58 59 51

7/09 86/79 78 70

7/10 73/72 70 59

7/11 56/55 25 24

7/12 47/38 61 57

7/13 51/49 52 46

7/14 54/49 52 45

7/15 60/54 61 54

7/16 44/41 40 37

7/17 49/47 34 31

7/18 51/47 54 39

7/19 56/50 50 38

7/20 74/68 57 47

7/21 92/79 67 50

7/22 79/64 77 70

7/23 71/65 64 61

7/24 95/92 94 80

7/25 MM 89 78

7/26 114/96 103 92

7/27 MM 68 61

7/28 MM 54 47

7/29 52/48 51 47

7/30 MM 59 52

7/31 66/64 58 52

Date CMAQ Forecast (1/8) Monitor. Results (1) Monitor. Results (8)

8/01 74/72 61 56

8/02 80/77 72 68

8/03 87/81 79 71

8/04 103/95 96 74

8/05 93/86 78 72

8/06 105/95 91 66

8/07 54/52 30* 35*

8/08 54/50 38 30

8/09 65/56 54 43

8/10 84/78 68 61

8/11 85/80 94 76

8/12 89/86 102 78

8/13 93/83 78 62

8/14 98/91 74 51

8/15 87/74 88 64

8/16 79/70 74 60

8/17 80/72 59 46

8/18 54/48 63 40

8/19 57/52 76 63

8/20 51/48 85 71

8/21 77/70 59 54

8/22 MM 74 61

8/23 86/67 62 53

8/24 77/75 71 63

8/25 73/57 82 61

8/26 62/55 73 63

8/27 72/67 76 62

8/28 66/63 70 60

8/29 61/55 50 45

8/30 43/39 42 35

8/31 70/67 69 62

*since 8-hour average exceeded 1-hour average, data are suspect and were not used.

Date CMAQ Forecast (1/8) Monitor. Results (1) Monitor. Results (8)

9/01 83/70 73 65

9/02 83/71 67 65

9/03 MM 66 63

9/04 MM 64 58

9/05 MM 71 65

9/06 71/67 70 62

9/07 78/70 69 62

9/08 MM 67 62

9/09 84/75 75 72

9/10 91/84 84 76

9/11 98/89 81 75

9/12 78/72 84 71

9/13 84/79 79 75

9/14 84/82 100 83

9/15 88/71 77 69

9/16 58/51 48 38

9/17 65/60 64 56

9/18 63/59 75 60

9/19 87/77 98 78

9/20 73/61 100 80

9/21 91/82 93 80

9/22 76/72 83 71

9/23 87/74 69 58

9/24 MM 74 61

9/25 61/56 60 50

9/26 43/39 30 18

9/27 64/58 60 53

9/28 61/55 66 54

9/29 MM 46 41

9/30 76/66 56 48

Note – Bolded numbers indicate values exceeding the NAAQS limit for ozone

TABLE 2. NAM-CMAQ OZONE FORECAST VERIFICATION STUDY DATA

Summer 2005

NAM-CMAQ Forecasts - Percent Time of Departure from Monitoring Data in Various Categories

Forecast-Observed	1-hour	8-hour
(high bias)
> +16 ppb	16.2%	19.1%
+13 to +16 ppb	5.9%	6.6%
+10 to +12 ppb	6.6%	7.4%
+7 to +9 ppb	9.5%	11.8%
+4 to +6 ppb	8.1%	14.0%
+1 to +3 ppb	15.4%	11.8%
0 ppb	2.2%	2.9%
-1 to -3 ppb	7.4%	7.4%
-4 to -6 ppb	5.9%	5.1%
-7 to -9 ppb	5.9%	2.9%
-10 to -12 ppb	3.7%	3.7%
-13 to -16 ppb	6.6%	0.7%
< -16 ppb	6.6%	6.6%
(low bias)