Figure 1: Location Map of the Nine Model Comparison Sites
Two major field campaigns and the OTAG process have provided an
opportunity to compare regional air quality measurements of ozone,
total oxides of nitrogen (NOy), and carbon monoxide (CO) with
UAM-V model output at nine monitoring stations. These stations
were operational during the period July 7, 1995 through July 18,
1995 and their data are compared to the 1995 base "D2"
run of UAM-V. Figure 1 shows the location of each of the monitoring
stations in the OTAG domain. These monitoring stations, from the
Southern Oxidants Study and NARSTO-NE, were sited with the intent
to provide regionally representative data and are classified as
rural in location. All are sited in large clearings with excellent
fetch. Their rural locations and excellent site characteristics
make them excellent candidates for a comparison. All stations
collected ozone and NOy data and four of the stations also collected
CO data. Hourly values were compared. Measured values were all
taken at 10 meters above ground level and were compared to the
hourly average UAM-V output for the grid cell (12 kilometers in
all cases but one) that contained the monitoring station. Several
comparisons were made to attempt to answer the following questions.
Figure 1: Location Map of the Nine Model Comparison Sites
Typically, modeled values are compared to measured values where
the measured values are set as the independent variable or the
'truth'. In this comparison, however, the modeled values will
be treated as the independent variable in order to investigate
how well the model performs in a specific class of output, i.e.
high modeled ozone. The purpose of this model comparison is not
only to explore how well the model represents ozone and NOy at
regional locations overall but also to take a close look at what
biases in the model might substantively influence interpretation
of subsequent runs where emissions inventories are altered to
test various control strategies. This methodology makes more sense
when we realize that in subsequent model runs made to test the
efficacy of different emissions control strategies, the base case
output becomes the independent axis or the 'truth' against
which the subsequent runs are judged. It is also important to
note that the output data that will receive the most scrutiny
will be those ozone values in the base case that are at or above
the NAAQS for ozone. Therefore, we need to fully understand not
only how the model performs on average but also how well it performs
in this critical interpretive area, the area where the base case
predicts the highest concentrations of ozone.
Back to Contents
Back to Contents
Ozone vs. NOy as an Indicator of Chemistry
Performance
Field studies over the past 10 years have investigated the relationship
between NOy and ozone concentrations (Trainer, et al. 1994; Kleinman,
et al. 1994, Olszyna, et al. 1994). These studies have shown remarkable
consistency in the relationship at many different times of the
year and at locations throughout the OTAG domain. The slope of
the relationship is typically in the 6:1 to 10:1 range in these
measurements. Ozone and NOy from both the measured and modeled
data sets were graphed and analyzed statistically to see how well
the modeled data reproduced the observed relationship.
In order to isolate well-mixed conditions only data from hours
between 1100 LST and 1900 LST were selected from each data set.
Figure 2 shows a scatterplot of modeled ozone and NOy data from
all nine sites during the 1995 episode. Figure 3 shows a similar
scatterplot of the measured data. The modeled data show more scatter
and a larger number of NOy data points above 15 ppb than the measured
data. Both plots show increasing scatter above 10 ppb NOy.
Figure 2: Modeled Ozone Vs. NOy
Figure 3: Measured Ozone Vs. NOy
The more robust relationship, that between ozone and NOz (i.e.,
NOy - NOx, the reaction products of NOx), could not be examined
because these sites did not have NOx data. The ozone vs. NOy relationship
is not as strong as ozone vs. NOz because NO, a constituent of
NOy but not of NOz, is often negatively correlated with ozone.
When NO is a large contributor to NOy the slope of the ozone/NOy
relationship is decreased relative to the ozone/NOz slope. NO,
an emitted species, is a larger contributor to NOy when the air
mass in question is 'younger' or made up of a larger fraction
of recently emitted NOx. The NOx/NOy ratio can thus be considered
an indicator of overall chemical air mass age, i.e. the degree
to which emitted NOx has been processed into reaction products,
NOz.
The increased scatter in both plots above 10 ppb suggests that
when NOy concentrations reach these levels NO is more likely to
be a significant component of NOy. The increasing proportion of
NO/NOy as NOy increases is responsible for the 'flattening out'
of the ozone/NOy curve as higher NOy concentrations are reached.
The increased scatter in the modeled ozone/NOy relationship and
the higher number of modeled NOy values above 15 ppb may be an
indication that NOx is not being processed into NOz as rapidly
in the model as in the measurements. The similarity of the general
appearance of the plots, however, suggests that the chemistry
is not grossly incorrect.
Figures 4 and 5 show scatter plots of the same two data sets which
have been sorted by ozone concentration and segregated into ten
bins of equal numbers of data points (about 70 points in each
bin). The ozone and NOy values for each bin are averaged and the
resulting 10 data pairs are plotted and regressed. The slopes
of the two regressions are remarkably similar, 8.73 for the modeled
data vs. 9.23 for the measured data, and the intercepts are essentially
equal.
Figure 4: Modeled Ozone Vs. NOy
Figure 5: Measured Ozone Vs. NOy
The general ozone vs. NOy relationship in the model compares reasonably
well with that in the measured data and both relationships agree
reasonably well with previously reported values for ozone vs.
NOy. The increased scatter in the relationship in the model as
well as the larger number of NOy values above 15 ppb may, however,
be indicative of less rapid oxidation of NOx into NOz in the model
than in the measured data.
Back to Contents
Overall Comparison of Ozone, NOy, and CO
Measurements
Table 1: Summary Statistics on Modeled and Measured Ozone, NOy, and CO
The next stage of comparison was to estimate the overall agreement
of the datasets on ozone, NOy, and CO concentrations. Table 1
summarizes descriptive statistics on modeled and measured data.
Ozone and NOy agree reasonably well on average while modeled CO
shows a bias towards overprediction. The standard deviation in
modeled values is generally higher than that of the measured values,
especially in the case of CO, indicating a wider dynamic range
in concentrations. The standard deviation of NOy is particularly
large in both the measured and modeled data due to a higher frequency
of short term variations in concentrations. These frequent variations
are the result of the impact of plumes of NOy on the sites, plumes
whose NOy concentrations are often an order of magnitude higher
than median NOy concentrations.
Table 2 lists the average daily ozone residual by site and by date. The average daily ozone residual was calculated as the 24 hour average of the hour by hour difference between modeled ozone and measured ozone. A summary column also shows the average ozone residual for all sites by date. There is considerable variation in behavior day to day and between sites. There also seems to be a tendency towards a positive residual with 53 of 68 (or 78%) of daily residuals between 7/11/95 and 7/18/95 greater than zero. It should be noted that the first 2-3 days in the table are part of the model 'ramp-up' period where the influence of initial conditions is still strong.
Table 2: Average Daily Ozone Residual, Modeled Ozone - Measured Ozone (ppb) Sites are listed South to North
| 07/07/95 | -12 | -11 | -12 | -20 | -20 | -19 | -4 | 2 | -11 | |
| 07/08/95 | -9 | -1 | 0 | -17 | -17 | -14 | -12 | -10 | -1 | |
| 07/09/95 | -1 | 6 | 7 | 1 | -1 | -15 | -7 | 2 | 2 | |
| 07/10/95 | 0 | 11 | 20 | 6 | 10 | -11 | -2 | -2 | -8 | |
| 07/11/95 | 5 | 11 | 18 | -14 | 15 | -19 | 1 | 19 | 5 | 10 |
| 07/12/95 | 7 | 11 | 8 | 2 | 28 | -14 | -3 | 4 | 16 | 8 |
| 07/13/95 | 7 | 11 | 2 | 12 | 19 | -1 | -10 | 20 | 9 | 4 |
| 07/14/95 | 9 | 13 | -7 | 22 | 17 | -9 | 3 | 12 | 20 | |
| 07/15/95 | 7 | 14 | -2 | 15 | 19 | 3 | -7 | -6 | 22 | |
| 07/16/95 | 7 | -2 | 10 | 4 | 13 | 17 | 1 | 20 | 2 | -1 |
| 07/17/95 | 16 | 6 | 22 | -1 | 28 | 23 | 27 | 28 | 6 | 2 |
| 07/18/95 | 2 | 10 | 5 | 7 | 6 | 10 | -9 | 9 |
Back to Contents
Apparent Biases in Model Predictions
Finally, the data were analyzed for evidence of any specific biases.
One example was immediately evident. Examination of time-series
plots (among Figures 6 through 12) for the Oak Grove site showed
that model predicted values for all three species were much less
variable than the measured values. It is hypothesized that the
coarser grid scale and fewer model layers (see section on caveats)
are responsible. This bias is not unexpected but is a good indication
of the impact of model resolution on the accuracy of predictions.
Figure 6. Time Series Plot of Modeled and Measured Ozone
| Figure 6--Centreville, AL | Figure 6--Oak Grove, MS | Figure 6--Yorkville, GA | |
|---|---|---|---|
 |
 |
 |
Figure 7. Time Series Plot of Modeled and Measured Ozone
| Figure 7--Arendtsville, PA | Figure 7--Dickson, TN | Figure 7--Holbrook, PA |  |
 |
 |
|---|
Figure 8. Time Series Plot of Modeled and Measured Ozone
| Figure 8--Brookhaven, NY | Figure 8--Kunkletown, PA | Figure 8--Truro, MA |
|---|---|---|
 |
 |
 |
Figure 9. Time Series Plot of Modeled and Measured Ozone
| Figure 9--Centreville, AL | Figure 9--Oak Grove, MS | Figure 9--Yorkville, GA |
|---|---|---|
 |
 |
 |
Figure 10. Time series Plot of Modeled and Predicted NOyP>
| Figure 10--Arendtsville, PA | Figure 10--Dickson, TN | Figure 10--Holbrook, PA |
|---|---|---|
 |
 |
 |
Figure 11. Time Series Plot of Modeled and Measured NOyP>
| Figure 11--Brookhaven, NY | Figure 11--Kunkletown, PA | Figure 11--Truro, MA |
|---|---|---|
 |
 |
 |
Figure 12. Time Series Plot of Modeled and Measured CO
| Figure 12--Arendtsville, PA | Figure 12--Oak Grove, MS | Figure 12--Yorkville, GA |
|---|---|---|
 |
 |
 |
The time-series plots also seem to show several cases of overprediction
of ozone concentrations, particularly when model predicted values
were above 100 ppb. Time-series plots of CO data also seem to
show significant overpredictions of CO concentrations, especially
at the Yorkville, GA and Dickson, TN sites although the only northern
site with CO, Arendtsville, PA, did show some overprediction late
in the episode.
Figures 13 through 15 focus on the model overprediction of all
species under high model predicted ozone concentration.
Again, the model output has been presented as the independent
variable in these comparisons because in subsequent examinations
of emission strategies runs the base case will be treated as the
benchmark. For these figures, all data pairs were sorted on decreasing
model predicted value for the species of interest. The sorted
data were then divided into 10 equal bins of data. For each bin
the average model predicted value was calculated and the average
hourly residual (model predicted value minus measured value) was
calculated. These figures present the average modeled value for
the bin increasing to the right along the x-axis and the average
residual for each bin along the y-axis.
Figure 13
Figure 14
Figure 15
All three figures show a similar pattern with increasingly large
residuals as the model predicted value increases. The overprediction
in CO concentrations is most dramatic with modeled concentrations
two are three times higher than measured concentrations. Overpredictions
in ozone values are not as severe but are of concern because they
occur at high model ozone concentrations where they could possibly
move the daily maximum ozone value across the NAAQS threshold.
Both northern (NARSTO-NE) sites and southern (Southern Oxidants
Study) sites show the same relationship when examined seperately.
The CO data may give some insight into the cause of these overpredictions.
CO has a relatively long atmospheric lifetime (~20-30 days), has
a very low deposition velocity, and is not produced rapidly by
ozone chemistry. For these reasons, CO concentrations are not
expected to build up rapidly on a regional scale. Two possible
reasons for the CO buildup in the model come to mind. Since CO
is neither rapidly produced chemically or deposited out physically
the CO buildup could be indicative of a vertical diffusivity problem
that is not as obvious in the more variable ozone and NOy data.
CO emissions inventories were also not as carefully prepared as
VOC and NOx emissions and the overprediction could be a result
of inaccurate emissions data.
Figure 16 presents ozone comparison data in a similar fashion
to Figure 13 but, in this case, the ozone values are sorted on
descending measured concentration rather than modeled concentration.
In this figure the overprediction of ozone seems to disappear.
This change in the average residual occurs because, when the ozone
data are sorted on measured concentration, the model prediction
for the top 240 measured concentrations is distributed roughly
evenly about the mean measured value. This distribution results
in a slight underprediction in concentration on average but with
a high degree of scatter about the mean.
The difference between sorting techniques brings forward an important
point when considering model results. Sorting on model output
will tend to focus one's attention on any biases evident from
the model point-of-view. If the data are sorted on measured
values ones' attention is drawn to biases evident from the measurement
point-of-view. This distinction becomes more meaningful when one
considers the ultimate use of model runs. A series of emission
control cases will be run and the efficacy of each of those cases
will be judged on the ozone concentration field, particularly
the highest values, that the model base case produces.
Because of this approach it becomes important to understand how
the model behaves when it is predicting high ozone values and
this analysis has attempted to do that.
Figure 17
Figure 18
Figures 17 and 18 present the subjective impact of such model
behavior. In Figure 17 it is shown that the model predicts three
times as many hours of ozone concentrations greater than 120 ppb
and similarly overpredicts the number of hours above the 100 ppb
and 80 ppb thresholds. Figure 18 shows that this model behavior
results in 3 more (nearly a doubling of) exceedences of the NAAQS
than were actually observed.
Back to Contents
In summary, intensive field measurements and the OTAG process
have provided a unique opportunity for comparison of regional
air quality measurements against the latest generation of air
quality modeling. A comparison of the ozone vs. NOy relationship
between the modeled ozone and NOy data and the measured ozone
and NOy data have shown the two data sets to be in good agreement
with one another as well as with expectations from previous field
studies. This good agreement is taken as an indication that there
are no gross errors in the ozone chemistry in the model. An overall
comparison of ozone, NOy and CO data shows that, on average, ozone
and NOy are well predicted but that CO is generally overpredicted,
at least in this data set.
A closer review of the data set reveals some areas of possible
bias. The Oak Grove model data, coming from an area of coarser
resolution in the model, showed significantly less variability
than did the measured data although the general trends were reasonably
well represented. This is not a surprising finding but is a good
indicator of model sensitivity to grid scale and number of layers.
The apparent lack of gross error in the ozone - NOy chemistry
coupled with the overpredictions of CO suggest that problems may
exist in the emissions or meteorology portions of the model. Also,
the larger degree of scatter in the modeled ozone/NOy relationship
along with the larger number of NOy concentrations greater the
15 ppb may indicate that the model is not processing NOx into
reaction products, NOz, as rapidly as is happening in the atmosphere.
Review of time-series data suggests that the model might be overpredicting
ozone under high modeled ozone conditions. The suggestion was
borne out when model performance for the top 10 percent of modeled
results for each species was examined. For all three species the
model tended to overpredict the top 10 percent of modeled values.
This overprediction potentially has policy implications if modeled
ozone values are erroneously high.
Disclaimer: Data for this analysis were provided by the Southern
Oxidants Study and NARSTO-NE (measured data) and by SAI, Inc.
(modeled data). The views presented here are those of the authors
and are not necessarily those of the Southern Oxidants Study,
NARSTO-NE, or SAI, Inc. This work was funded by Southern Company
Services.
Back to Contents
Kleinman, Lawrence, Yin-Nan Lee, Stephen R. Springston, Linda
Nunnermacker, Xianliang Zhou, Robert Brown, Kristen Hallock, Paul
Klotz, Daniel Leahy, Jai H. Lee, and Leonard Newman. Ozone formation
at a rural site in the southeastern United States. J. Geophys.
Res. 99(D2), 3469-3482. 1994
Olszyna, Kenneth J., Elizabeth M. Bailey, Romualdas Simonaitis,
and James F. Meagher. O3 and NOy relationships at a rural site.
J. Geophys. Res. 99(D7), 14557-14563, 1994.
Trainer, M., D. D. Parrish, M. P. Buhr, R. B. Norton, F. C. Fehsenfeld,
K. G. Anlauf, J. W. Bottenheim, Y. Z. Tang, H. A. Wiebe, J. M.
Roberts, R. L. Tanner, L. Newman, V. C. Bowersox, J. B. Meagher,
K. J. Olszyna, M. O. Rodgers, T. Want, H. Berresheim, K. L. Demerjian,
And U. K. Roychowdhury. Correlation of ozone with NOy in photochemically
aged air. J. Geophys. Res. 98, pp 2917-2925. 1993.
Mueller, P. K and P. T. Roberts. Memorandum to the NARSTO-NE Quality
Systems Team. Data Qualification Statement, 1995 Surface Air Quality
and Meteorological Data: Environmental Science and Engineering,
Version 3, September 19, 1996 (draft).
Olszyna, K. J. Performance Audit Report for the Southeastern Cosortium:
Intermediate Oxidants Network: Yorkville. June 19-20, 1996 (draft).
Back to Contents
| Submit your comments, feedback, questions, and ideas pertaining this page. Your input will be automatically added to the existing annotations. In order to add a new comment, you must be registered with the OTAG/AQA Peoples Page. |
