Draft Technical Report

A Comparison of Modeled and Measured Ozone, NOy and CO at Nine Regional Monitoring Stations during the 1995 OTAG Episode

Ben Hartsell, ESE Environmental

Eric Edgerton, ESE Environmental



Two major field campaigns and the OTAG process have provided an opportunity to compare regional air quality measurements of ozone, total oxides of nitrogen (NOy), and carbon monoxide (CO) with UAM-V model output at nine monitoring stations. These stations were operational during the period July 7, 1995 through July 18, 1995 and their data are compared to the 1995 base "D2" run of UAM-V. Figure 1 shows the location of each of the monitoring stations in the OTAG domain. These monitoring stations, from the Southern Oxidants Study and NARSTO-NE, were sited with the intent to provide regionally representative data and are classified as rural in location. All are sited in large clearings with excellent fetch. Their rural locations and excellent site characteristics make them excellent candidates for a comparison. All stations collected ozone and NOy data and four of the stations also collected CO data. Hourly values were compared. Measured values were all taken at 10 meters above ground level and were compared to the hourly average UAM-V output for the grid cell (12 kilometers in all cases but one) that contained the monitoring station. Several comparisons were made to attempt to answer the following questions.

Figure 1: Location Map of the Nine Model Comparison Sites

Typically, modeled values are compared to measured values where the measured values are set as the independent variable or the 'truth'. In this comparison, however, the modeled values will be treated as the independent variable in order to investigate how well the model performs in a specific class of output, i.e. high modeled ozone. The purpose of this model comparison is not only to explore how well the model represents ozone and NOy at regional locations overall but also to take a close look at what biases in the model might substantively influence interpretation of subsequent runs where emissions inventories are altered to test various control strategies. This methodology makes more sense when we realize that in subsequent model runs made to test the efficacy of different emissions control strategies, the base case output becomes the independent axis or the 'truth' against which the subsequent runs are judged. It is also important to note that the output data that will receive the most scrutiny will be those ozone values in the base case that are at or above the NAAQS for ozone. Therefore, we need to fully understand not only how the model performs on average but also how well it performs in this critical interpretive area, the area where the base case predicts the highest concentrations of ozone.

Back to Contents


There are several caveats to such a comparison that must be understood. Back to Contents

Ozone vs. NOy as an Indicator of Chemistry Performance

Field studies over the past 10 years have investigated the relationship between NOy and ozone concentrations (Trainer, et al. 1994; Kleinman, et al. 1994, Olszyna, et al. 1994). These studies have shown remarkable consistency in the relationship at many different times of the year and at locations throughout the OTAG domain. The slope of the relationship is typically in the 6:1 to 10:1 range in these measurements. Ozone and NOy from both the measured and modeled data sets were graphed and analyzed statistically to see how well the modeled data reproduced the observed relationship.

In order to isolate well-mixed conditions only data from hours between 1100 LST and 1900 LST were selected from each data set. Figure 2 shows a scatterplot of modeled ozone and NOy data from all nine sites during the 1995 episode. Figure 3 shows a similar scatterplot of the measured data. The modeled data show more scatter and a larger number of NOy data points above 15 ppb than the measured data. Both plots show increasing scatter above 10 ppb NOy.

Figure 2: Modeled Ozone Vs. NOy

Figure 3: Measured Ozone Vs. NOy

The more robust relationship, that between ozone and NOz (i.e., NOy - NOx, the reaction products of NOx), could not be examined because these sites did not have NOx data. The ozone vs. NOy relationship is not as strong as ozone vs. NOz because NO, a constituent of NOy but not of NOz, is often negatively correlated with ozone. When NO is a large contributor to NOy the slope of the ozone/NOy relationship is decreased relative to the ozone/NOz slope. NO, an emitted species, is a larger contributor to NOy when the air mass in question is 'younger' or made up of a larger fraction of recently emitted NOx. The NOx/NOy ratio can thus be considered an indicator of overall chemical air mass age, i.e. the degree to which emitted NOx has been processed into reaction products, NOz.

The increased scatter in both plots above 10 ppb suggests that when NOy concentrations reach these levels NO is more likely to be a significant component of NOy. The increasing proportion of NO/NOy as NOy increases is responsible for the 'flattening out' of the ozone/NOy curve as higher NOy concentrations are reached. The increased scatter in the modeled ozone/NOy relationship and the higher number of modeled NOy values above 15 ppb may be an indication that NOx is not being processed into NOz as rapidly in the model as in the measurements. The similarity of the general appearance of the plots, however, suggests that the chemistry is not grossly incorrect.

Figures 4 and 5 show scatter plots of the same two data sets which have been sorted by ozone concentration and segregated into ten bins of equal numbers of data points (about 70 points in each bin). The ozone and NOy values for each bin are averaged and the resulting 10 data pairs are plotted and regressed. The slopes of the two regressions are remarkably similar, 8.73 for the modeled data vs. 9.23 for the measured data, and the intercepts are essentially equal.

Figure 4: Modeled Ozone Vs. NOy

Figure 5: Measured Ozone Vs. NOy

The general ozone vs. NOy relationship in the model compares reasonably well with that in the measured data and both relationships agree reasonably well with previously reported values for ozone vs. NOy. The increased scatter in the relationship in the model as well as the larger number of NOy values above 15 ppb may, however, be indicative of less rapid oxidation of NOx into NOz in the model than in the measured data.

Back to Contents

Overall Comparison of Ozone, NOy, and CO Measurements

Table 1: Summary Statistics on Modeled and Measured Ozone, NOy, and CO

Model O3
Meas O3
Model NOy
Meas NOy
Model CO
Meas CO
55 ppb
52 ppb
7.30 ppb
6.35 ppb
221 ppb
163 ppb
5 ppb
2 ppb
0.12 ppb
0.34 ppb
98 ppb
60 ppb
157 ppb
138 ppb
63.8 ppb
72.93 ppb
607 ppb
388 ppb
St. Dev.
24 ppb
22 ppb
6.95 ppb
5.47 ppb
96 ppb
46 ppb
52 ppb
50 ppb
5.33 ppb
4.99 ppb
194 ppb
156 ppb
Geo. Mean
35 ppb
36 ppb
2.77 ppb
3.66 ppb
203 ppb
157 ppb
Valid n

The next stage of comparison was to estimate the overall agreement of the datasets on ozone, NOy, and CO concentrations. Table 1 summarizes descriptive statistics on modeled and measured data. Ozone and NOy agree reasonably well on average while modeled CO shows a bias towards overprediction. The standard deviation in modeled values is generally higher than that of the measured values, especially in the case of CO, indicating a wider dynamic range in concentrations. The standard deviation of NOy is particularly large in both the measured and modeled data due to a higher frequency of short term variations in concentrations. These frequent variations are the result of the impact of plumes of NOy on the sites, plumes whose NOy concentrations are often an order of magnitude higher than median NOy concentrations.

Table 2 lists the average daily ozone residual by site and by date. The average daily ozone residual was calculated as the 24 hour average of the hour by hour difference between modeled ozone and measured ozone. A summary column also shows the average ozone residual for all sites by date. There is considerable variation in behavior day to day and between sites. There also seems to be a tendency towards a positive residual with 53 of 68 (or 78%) of daily residuals between 7/11/95 and 7/18/95 greater than zero. It should be noted that the first 2-3 days in the table are part of the model 'ramp-up' period where the influence of initial conditions is still strong.

Table 2: Average Daily Ozone Residual, Modeled Ozone - Measured Ozone (ppb) Sites are listed South to North

07/07/95 -12 -11 -12 -20 -20 -19 -4 2 -11
07/08/95 -9 -1 0 -17 -17 -14 -12 -10 -1
07/09/95 -1 6 71 -1 -15 -7 2 2
07/10/95 0 11 20 610 -11 -2 -2 -8
07/11/95 5 11 18 -14 15 -19 119 5 10
07/12/95 7 11 82 28 -14 -3 4 16 8
07/13/95 7 11 212 19 -1 -10 20 9 4
07/14/95 9 13 -7 22 17 -9 3
12 20
07/15/95 7 14 -2 15 19 3-7
-6 22
07/16/95 7 -2 10 413 17 1 20 2-1
07/17/95 16 6 22 -1 28 23 27 28 62
07/18/95 2 10 57 6
-9 9

Back to Contents

Apparent Biases in Model Predictions

Finally, the data were analyzed for evidence of any specific biases. One example was immediately evident. Examination of time-series plots (among Figures 6 through 12) for the Oak Grove site showed that model predicted values for all three species were much less variable than the measured values. It is hypothesized that the coarser grid scale and fewer model layers (see section on caveats) are responsible. This bias is not unexpected but is a good indication of the impact of model resolution on the accuracy of predictions.

Figure 6. Time Series Plot of Modeled and Measured Ozone

Figure 6--Centreville, AL Figure 6--Oak Grove, MS Figure 6--Yorkville, GA

Figure 7. Time Series Plot of Modeled and Measured Ozone

Figure 7--Arendtsville, PA Figure 7--Dickson, TN Figure 7--Holbrook, PA

Figure 8. Time Series Plot of Modeled and Measured Ozone

Figure 8--Brookhaven, NY Figure 8--Kunkletown, PA Figure 8--Truro, MA

Figure 9. Time Series Plot of Modeled and Measured Ozone

Figure 9--Centreville, AL Figure 9--Oak Grove, MS Figure 9--Yorkville, GA

Figure 10. Time series Plot of Modeled and Predicted NOyP>
Figure 10--Arendtsville, PA Figure 10--Dickson, TN Figure 10--Holbrook, PA

Figure 11. Time Series Plot of Modeled and Measured NOyP>
Figure 11--Brookhaven, NY Figure 11--Kunkletown, PA Figure 11--Truro, MA

Figure 12. Time Series Plot of Modeled and Measured CO

Figure 12--Arendtsville, PA Figure 12--Oak Grove, MS Figure 12--Yorkville, GA

The time-series plots also seem to show several cases of overprediction of ozone concentrations, particularly when model predicted values were above 100 ppb. Time-series plots of CO data also seem to show significant overpredictions of CO concentrations, especially at the Yorkville, GA and Dickson, TN sites although the only northern site with CO, Arendtsville, PA, did show some overprediction late in the episode.

Figures 13 through 15 focus on the model overprediction of all species under high model predicted ozone concentration. Again, the model output has been presented as the independent variable in these comparisons because in subsequent examinations of emission strategies runs the base case will be treated as the benchmark. For these figures, all data pairs were sorted on decreasing model predicted value for the species of interest. The sorted data were then divided into 10 equal bins of data. For each bin the average model predicted value was calculated and the average hourly residual (model predicted value minus measured value) was calculated. These figures present the average modeled value for the bin increasing to the right along the x-axis and the average residual for each bin along the y-axis.

Figure 13

Figure 14

Figure 15

All three figures show a similar pattern with increasingly large residuals as the model predicted value increases. The overprediction in CO concentrations is most dramatic with modeled concentrations two are three times higher than measured concentrations. Overpredictions in ozone values are not as severe but are of concern because they occur at high model ozone concentrations where they could possibly move the daily maximum ozone value across the NAAQS threshold. Both northern (NARSTO-NE) sites and southern (Southern Oxidants Study) sites show the same relationship when examined seperately.

The CO data may give some insight into the cause of these overpredictions. CO has a relatively long atmospheric lifetime (~20-30 days), has a very low deposition velocity, and is not produced rapidly by ozone chemistry. For these reasons, CO concentrations are not expected to build up rapidly on a regional scale. Two possible reasons for the CO buildup in the model come to mind. Since CO is neither rapidly produced chemically or deposited out physically the CO buildup could be indicative of a vertical diffusivity problem that is not as obvious in the more variable ozone and NOy data. CO emissions inventories were also not as carefully prepared as VOC and NOx emissions and the overprediction could be a result of inaccurate emissions data.

Figure 16 presents ozone comparison data in a similar fashion to Figure 13 but, in this case, the ozone values are sorted on descending measured concentration rather than modeled concentration. In this figure the overprediction of ozone seems to disappear. This change in the average residual occurs because, when the ozone data are sorted on measured concentration, the model prediction for the top 240 measured concentrations is distributed roughly evenly about the mean measured value. This distribution results in a slight underprediction in concentration on average but with a high degree of scatter about the mean.

The difference between sorting techniques brings forward an important point when considering model results. Sorting on model output will tend to focus one's attention on any biases evident from the model point-of-view. If the data are sorted on measured values ones' attention is drawn to biases evident from the measurement point-of-view. This distinction becomes more meaningful when one considers the ultimate use of model runs. A series of emission control cases will be run and the efficacy of each of those cases will be judged on the ozone concentration field, particularly the highest values, that the model base case produces. Because of this approach it becomes important to understand how the model behaves when it is predicting high ozone values and this analysis has attempted to do that.

Figure 17

Figure 18

Figures 17 and 18 present the subjective impact of such model behavior. In Figure 17 it is shown that the model predicts three times as many hours of ozone concentrations greater than 120 ppb and similarly overpredicts the number of hours above the 100 ppb and 80 ppb thresholds. Figure 18 shows that this model behavior results in 3 more (nearly a doubling of) exceedences of the NAAQS than were actually observed.

Back to Contents

Summary and Conclusions

In summary, intensive field measurements and the OTAG process have provided a unique opportunity for comparison of regional air quality measurements against the latest generation of air quality modeling. A comparison of the ozone vs. NOy relationship between the modeled ozone and NOy data and the measured ozone and NOy data have shown the two data sets to be in good agreement with one another as well as with expectations from previous field studies. This good agreement is taken as an indication that there are no gross errors in the ozone chemistry in the model. An overall comparison of ozone, NOy and CO data shows that, on average, ozone and NOy are well predicted but that CO is generally overpredicted, at least in this data set.

A closer review of the data set reveals some areas of possible bias. The Oak Grove model data, coming from an area of coarser resolution in the model, showed significantly less variability than did the measured data although the general trends were reasonably well represented. This is not a surprising finding but is a good indicator of model sensitivity to grid scale and number of layers.

The apparent lack of gross error in the ozone - NOy chemistry coupled with the overpredictions of CO suggest that problems may exist in the emissions or meteorology portions of the model. Also, the larger degree of scatter in the modeled ozone/NOy relationship along with the larger number of NOy concentrations greater the 15 ppb may indicate that the model is not processing NOx into reaction products, NOz, as rapidly as is happening in the atmosphere.

Review of time-series data suggests that the model might be overpredicting ozone under high modeled ozone conditions. The suggestion was borne out when model performance for the top 10 percent of modeled results for each species was examined. For all three species the model tended to overpredict the top 10 percent of modeled values. This overprediction potentially has policy implications if modeled ozone values are erroneously high.

Disclaimer: Data for this analysis were provided by the Southern Oxidants Study and NARSTO-NE (measured data) and by SAI, Inc. (modeled data). The views presented here are those of the authors and are not necessarily those of the Southern Oxidants Study, NARSTO-NE, or SAI, Inc. This work was funded by Southern Company Services.

Back to Contents


Kleinman, Lawrence, Yin-Nan Lee, Stephen R. Springston, Linda Nunnermacker, Xianliang Zhou, Robert Brown, Kristen Hallock, Paul Klotz, Daniel Leahy, Jai H. Lee, and Leonard Newman. Ozone formation at a rural site in the southeastern United States. J. Geophys. Res. 99(D2), 3469-3482. 1994

Olszyna, Kenneth J., Elizabeth M. Bailey, Romualdas Simonaitis, and James F. Meagher. O3 and NOy relationships at a rural site. J. Geophys. Res. 99(D7), 14557-14563, 1994.

Trainer, M., D. D. Parrish, M. P. Buhr, R. B. Norton, F. C. Fehsenfeld, K. G. Anlauf, J. W. Bottenheim, Y. Z. Tang, H. A. Wiebe, J. M. Roberts, R. L. Tanner, L. Newman, V. C. Bowersox, J. B. Meagher, K. J. Olszyna, M. O. Rodgers, T. Want, H. Berresheim, K. L. Demerjian, And U. K. Roychowdhury. Correlation of ozone with NOy in photochemically aged air. J. Geophys. Res. 98, pp 2917-2925. 1993.

Mueller, P. K and P. T. Roberts. Memorandum to the NARSTO-NE Quality Systems Team. Data Qualification Statement, 1995 Surface Air Quality and Meteorological Data: Environmental Science and Engineering, Version 3, September 19, 1996 (draft).

Olszyna, K. J. Performance Audit Report for the Southeastern Cosortium: Intermediate Oxidants Network: Yorkville. June 19-20, 1996 (draft).

Back to Contents

Submit your comments, feedback, questions, and ideas pertaining this page. Your input will be automatically added to the existing annotations. In order to add a new comment, you must be registered with the OTAG/AQA Peoples Page.