Preparation of Ozone Files for Data Analysis and Model Comparison

Support Activity for the OTAG Air Quality and Modeling Workgroups

Submitted by:

Janja D. Husar and Rudolf B. Husar

Center for Air Pollution Impact and Trend Analysis (CAPITA)
Campus Box 1124
Washington University
St. Louis, MO 63130


Submitted to:

Neil Frank, Project Officer

Cooperative Agreement #CX 824179-01
Office of Air Quality Planning and Standards
U.S. Environmental Protection Agency
Research Triangle Park, NC

September 20, 1996

Contents:


Summary

The Urban Airshed Model (UAM-V) is being operated in support of the OTAG policy making process. The regional ozone model will be evaluated using air quality data. The purpose of this activity was to supply the necessary air quality data (O3, NOx, NO, NO2, NMHC) for model evaluation. The activities included (1) acquisition of ozone related air quality data from six different data sources and (2) preparation of these air quality data for use by the Air Quality and Modeling Workgroups.

The supplied data sets are from AIRS, CASTNet, EMEFS, SCION, LADCO, and Georgia. The data sets were supplied to the UAM-V Modeling community in a batch using AIRS data and another batch containing the combined data from all other sources. The data in each batch are provided in two versions. The first includes the complete or available data from a given network, and the second version includes only a subset of monitoring stations that were explicitly requested by the Modeling Workgroup. All the raw and re-formatted air quality data are shared through the OTAG Air Quality Analysis website.

Back to Contents


Background for the Activity

The Urban Airshed Model Version (UAM-V) will be applied by the OTAG to help identify and recommend a strategies to reduce transported ozone and its precursors which, in combination with other measures, will enable attainment and maintenance of the National Ambient Quality Standards for ozone in the eastern United States. Application of UAM-V will include an evaluation of model performance. The goal of this activity is to (1) acquire ozone related air quality data from six different data sources and (2) prepare of these air quality data for ingestion into the UAM-V model.

Back to Contents


Description of the Data Sets

Air quality data for OTAG/UAM-V Model evaluation has been collected from multiple sources:

Data SetSupplying Organization Years
AIRSEPA1988, 91, 93, 95
CASTNetEPA1988, 91, 93, 95
EMEFSEulerian Model Evaluation and Field Study 1988
SCIONSouthern Oxidant Study 1993, 95
LADCOLake Michigan Air Directors Consortium 1991 (88, 93, 95)
GEORGIAState of Georgia 1988, 91, 93, 95
FLORIDAState of Florida


Back to Contents


Required Output Data

The data structure and format required by UAM-V consist of:

  1. List of site locations, including latitude, longitude,
  2. Records describing the variables, including units
  3. List of 24 hourly data points.

The last two entries are repeated for each parameter and each monitoring site. The data for one day are placed in a file. Hence, an air quality data set consists of 48 files, one for each modeling day. An example partial output is shown in Table 1.

Table 1. Format required by UAM-V

116

051191002 -92.25 34.83

220730002 -92.08 32.51

291370001 -91.78 39.47

220331001 -91.20 30.58

551230008 -90.56 43.70

290990012 -90.36 38.43

051191002 CO 910721 PPB

-999.00 -999.00 -999.00 -999.00 -999.00 -999.00 -999.00

-999.00 -999.00 -999.00 -999.00 -999.00 -999.00 -999.00

-999.00 -999.00 -999.00 -999.00 -999.00 -999.00 -999.00

-999.00 -999.00 -999.00

051191002 NO 910721 PPB

-999.00 2.00 2.00 2.00 0.00 0.00 3.00

2.00 2.00 3.00 3.00 2.00 0.00 0.00

0.00 0.00 2.00 2.00 3.00 7.00 6.00

6.00 5.00 5.00

051191002 NO2 910721 PPB

-999.00 4.00 3.00 3.00 4.00 6.00 5.00

4.00 3.00 2.00 2.00 3.00 5.00 7.00

8.00 9.00 8.00 9.00 8.00 6.00 10.00

10.00 18.00 11.00

051191002 NOX 910721 PPB

-999.00 6.00 5.00 5.00 4.00 6.00 8.00

6.00 5.00 5.00 5.00 5.00 5.00 7.00

8.00 9.00 10.00 11.00 11.00 13.00 16.00

16.00 23.00 16.00

051191002 O3 910721 PPB

-999.00 12.00 12.00 13.00 15.00 13.00 10.00

11.00 14.00 15.00 18.00 25.00 34.00 48.00

47.00 40.00 37.00 41.00 40.00 36.00 28.00

21.00 11.00 11.00

Data from each network was to be extracted to include only the OTAG modeling periods

  • July 1-15, 1988,
  • July 13-21, 19991
  • July 20-30, 1993
  • July 7-18, 1995.

  • Back to Contents

    Data Translation Procedure

    The data translation has occurred in three steps. All of the input data from the different networks were first translated into a common set of relational tables in ASCII format. The second step consisted of compiling the ASCII tables into the binary Voyager format. The third step for UAM-V ingestion data from Voyager format were exported into daily files as specified by the UAM-V input specifications. The actual translation from Voyager to UAM-V format was accomplished using the scripting language of Voyager.

    The raw input data from individual networks, ASCII tables that were derived from the input data, and their associated data dictionaries, the resulting Voyager files, as well as the UAM-V input files are all accessible from the CAPITA FTP/WWW server.

    Back to Contents


    AIRS

    The Aerometric Information Retrieval System (AIRS) network provides the most comprehensive data for the eastern US. It provides the sensory information that is used in the regulatory feedback. Ozone exceedances of the National Air Quality Standard are based on the AIRS data.

    The AIRS data set consists of about 600 ozone monitoring sites east of the Rockies. At any given time hourly data are available for about 400 sites. Station density is highest in populated areas, but non-urban sites are also abundant. The AIRS ozone-related data were extracted from the AIRS at EPA as raw hourly data, using AMP350 workfiles as output.

    The extracted AIRS work files were downloaded from the EPA mainframe through FTP. The workfiles were then transformed into a set of relational tables containing the data, site locations, and parameter descriptions. These relational tables were then compiled into a Voyager format for verification. The AIRS workfiles, the ASCII tables that were derived from the work files, and their associated data dictionaries, as well as the resulting Voyager files are accessible from the CAPITA FTP/WWW server.

    Back to Contents


    CASTNet

    The CASTNet network was set up by EPA for long-term monitoring and emission tracking purposes.

    The CASTNet monitoring network measured ozone for about 70 eastern US locations, including some meteorological variables. The CASTNet network began the operation in 1987. CASTNet data were received at CAPITA from EPA on 2 CD ROMS and the entire data set (>600 MB) has been compiled into Voyager format to store the data more conveniently and efficiently.

    Original Data

    Data were recieved on 2 CD-ROMs. The CD-ROMs are stored in the OTAG folder. The readme file accompanying the 2 CD ROMS read as follows:

    This disk contains 5 files, this information file, 2 data files and 2 data definition files. The data files contain continuous meteorological (including ozone) and dry chemistry data for all available NDDN/CASTNet sites from 1/1/1987 through 9/30/1995. The three data definition files describe the format of the data files.

    Please contact Mrs. Linda Carmichael at ESE Environmental, Inc. if you need any additional information.

  • ESE Environmental, Inc.
  • 4915 Prospectus Drive
  • Suite J
  • Durham, NC 27713
  • (919) 544-3903
  • (919) 544-3882 (fax)
  • Data Manipulation

    The original data were manipulated at CAPITA and the following directories were created in the 5-step process.

    1_DataSource:

    The files obtained on the 2 CD-ROMs were to large thus only TXT files were transferred to this directory: . The latitude longitude for several sites had to be obtained from the CASTNet data managers.

  • 105DRY.TXT original chemical data in TXT format
  • CASTNET.DOC Correspondence regarding original data
  • DEFDRY.TXT description of chemical data
  • METDEF.TXT description of meteorological data
  • readme.TXT CAPITA listing of files
  • READMEO.TXT Raw data description that came with the CD ROMs.
  • 2_VoySource:

    The complete raw data received on CD ROMs were translated into Voyager.

  • Documentation: No files in this directory
  • Translation: CASTNET.LDB, CASTNET.MDB, NDDNLCT.XLS ((location table for NDDN (part of CASTNET) data set), README.TXT, UNIQLCT.XLS (unique locations from CASTNET network)
  • VoyDat: CASNET.WKB, CASTLCT.XLS, CASTNDT.ZIP, CASTNET.DAD, CASTNET.DAH, CASTNET.DAT, CASTNET.LCD, CASTNET.LCT, CASTNET.RPT, CASTNET.VOY, CASTNET.VRD, CASTNET.VRT, CASTNVY.ZIP CONTAINS CASTNET.VOY FILE, CN9CHLOCCODE.LCT
  • 3_VoyFixData.

    Data fixing was done in the CASTNET.VOY file. The *.gif files in the documentation record the problems detected with the five CASTNet stations.

  • Documentation: C_BVL130.GIF, C_BVL13Z.GIF, C_CNM167.GIF, C_CNM16Z.GIF, C_CVL151.GIF, C_CVL15Z.GIF, C_EGB181.GIF, C_EGB18Z.GIF, C_GCN174.BMP, C_GCN174.GIF, C_GCN17Z.BMP, C_GCN17Z.GIF, QUEDATA.DOC, QUEDATA.TXT, README.TXT, TRANSLATION, README.TXT, TMPCORRO3DATA, EXTRACT.FRM, EXTRACT.FRX, EXTRDAT.BAS, VAREXTR.VBP, VOYDAT, CASTNFIX.LDB, CASTNFIX.MDB, CASTNFIX.VOY, CASTNFDT.ZIP, README.TXT
  •         Network                 LocCode                 LocNAME                 Time                 Problem                 Action                 Reason        
    Castnet
    EGB181
    Egbert, Ontario
    Bad O3data
    Discard O3 from site
    Instrum malfuncti
    Castnet
    CVL151
    Coffeeville, MS
    17-Mar-1989 1400
    Spike
    Do nothing
    OK Data continuous, No AIRS
    Castnet
    CNM167
    Chiricahua, AZ
    9-May-1990 0400
    Spike
    Do nothing
    AIRS 040038001 has spike 135 ppb
    Castnet
    GCN174
    Grand Canyon, AZ
    13-Dec-1994 1300
    Spike
    Discard 1300 and 1400 hour data 3 and 5 ppb
    Castnet
    BVL130
    Bondville, IL
    14-Jun-1991 1200
    Spike
    Do nothing
    AIRS 170198001 has spike 145 ppb


    Figure 1 Figure 2 Figure 3 Figure 4 Figure 5
    Figure 6 Figure 7 Figure 8 Figure 9 Figure 10


    Back to Contents



    4_VoyO3.

    Ozone data were extracted from the files 3_VoyFixData.

  • Documentation: readme.TXT, DelRecZeroVal, LADCO.FRM, LADCO.frx, LADCO.vbp, LADCOTRN.BAS, VBO3CNExtr, O3CNExtr.BAS, O3CNExtr.FRM, O3CNExtr.frx, O3CNExtr.log, O3CNExtr.vbp
  • Translation: VBDelRecZeroVal - Visial Basic code to remove records with zero Ozone values in the O3CASTNET data set
  • VBO3CNExtr - Visial Basic code to extract noly Ozone data (woit Location code, date) from CASTNET data set
  • VoyDat: O3CASTdt.ZIP, O3CASTNT.dat, O3CASTNT.VOY(Voyager file with ozone data from CASTNET data set, spikes fixed, zeroes removed, location EGB181 excluded), O3CASTvy.ZIP (dat/dad, etc. files for voyager compilation), readme.TXT.
  • 5_VoyO3AgMx

    The ozone daily maximum data were aggregated from the 4_VoyO3 files.

  • Documentation:
  • Translation:
  • VoyDat: Daily maximum aggregated ozone data: O3AGCAST.DAD, O3AGCAST.DAH, O3AGCAST.DAT, O3AGCAST.LCD, O3AGCAST.lct - location table with excluded EGB181 location and LOC_NAME field formatted (spaces replaces by underscore, elevation included, length=62 ), O3AGCAST.PRP output table for CASTNET for Daily maximum aggregator, O3AGCAST.VOY, O3AGCAST.VRD, O3AGCAST.VRT, O3AMXCdt.ZIP dat/dad, etc. files for Ozone maximum daily aggregated data, O3AMXCvy.ZIP file for Ozone maximum daily aggregated data cotaining O3AGCAST.VOY file, O3CASTNT.PRP input table for CASTNET for aggregator, readme.TXT


  • EMEFS

    The purpose of the EMEFS network was to evaluate comprehensive regional Eulerian acid deposition models from US and Canada.

    The EMEFS data are available for 116 stations over the eastern US and Ontario, Canada. The network was operating between June 1988 and May, 1990. For most sites only ozone is available, but some sites have also NO2 data. The EMEFS network is composite of the following networks:

  • APIOS (OME)
  • CAPMon (AES)
  • FADMP (FCG)
  • MODES (TVA)
  • MODES-GRAD (EPA)
  • MODES- VAR (EPA)
  • NDDN (EPA)
  • OEN (EPRI).
  • 1_DataSource:

    The EMEFS data were obtained from the EMEFS data custodians, Earth Tech Inc. The data were transferred from the Earth Tech FTP site. The transferred data were zipped for storage. . The data documentation was sufficient to prepare the relational tables needed for Voyager, CONTCHEM.ZIP, EMESFNET.XLS, README.TXT

    2_VoySource.

    The complete raw data were translated into Voyager. Note that the raw data were reported in GMT (Greenwich Mean Time) time, thus the translated data files are still reflecting the GMT

  • Documentation: EMEFS.LDB, EMEFS.MDB, EMEFSMDB.ZIP, GMTDT.ZIP, GMTEMEFS.VOY, GMTVY.ZIP, README.TXT
  • Translation: The translation was standard and no routines for translation were written.
  • VoyDat: EMEFS.LDB, EMEFS.MDB, EMEFSMDB.ZIP, GMTDT.ZIP, GMTEMEFS.VOY, GMTVY.ZIP, README.TXT.
  • 3_VoyFixData.

    Data fixing was done in the CASTNET.VOY file. The *.GIF files in the documentation record the problems detected with the five CASTNet stations.

  • Documentation: The QA for ozone data was performed and the following questions for data validity were raised and actions implemented to correct the EMEFS data. The list of stations and data times of corrections were recorded in *.GIF FILES: E_08065B.GIF,
  • Translation: The routines using to change time to local standard time are given in the following files VBACCESSTIMESHIFT, VBcodeDateTimeShift,, vbSelDataForLoc, VBShiftTimeSelLoc, EMFSconc.cmd, EMFSconc.cmd
  • VoyDat: EMEFSFIX.LDB, readme.TXT, EMEFSSTVY.ZIP, EMLSTAll.VOY (without NDDN data (site Id =22)), EMLSTdt.ZIP, EMLSTmdb.ZIP, Emefsfix.mdb.
  •         Network                 LocCode                 LocNAME                 Time                 Problem                 Action                 Reason        
    EMEFS
    08531
    Marion, AL
    20-Aug-1988 2200;2300
    Spike
    Delete 2200;2300 hour points
    EMEFS
    08529
    Moorhead, KY
    20-Aug-1988 2300
    Spike
    Delete 2300 point
    EMEFS
    22560
    Deer Creek P, OH
    26-Jun-1989 1600
    Spike
    Do nothing
    AIRS 391298001 spike 132 ppb
    EMEFS
    08533
    34 due West, SC
    12-Sep-1988 1500
    Spike
    Delete 1500 hour point
    EMEFS
    08526
    Zanesville, OH
    05-Seo-1989 0800
    Spike
    Delete 0800 hour point
    EMEFS
    08065b
    Penn State, b, PA
    22-Jun-1988 1000,1100,1200
    Spike
    Delete 1000,1100,1200 hour points
    co-located no spike



    Figure 11 Figure 12 Figure 13 Figure 14 Figure 15 Figure 16
    Figure 17 Figure 18 Figure 19 Figure 20 Figure 21


    Back to Contents



    4_VoyO3

  • Documentation:
  • Translation: EMEFSLct.xls
  • VoyDat: O3ELSTdt.ZIP, O3ELSTvy.ZIP, O3EMLST.DAD, O3EMLST.DAH, O3EMLST.DAT, O3EMLST.LCD, O3EMLST.LCT, O3EMLST.VOY, O3EMLST.VRD, O3EMLST.VRT
  • 5_VoyO3AgMx

  • Documentation:
  • Translation:,
  • VoyDat: Files O3AMxEM1.* prepared for location table with EM22* location code sites: O3AMXEM1.LCT, O3AMXEM1.VOY, O3AMXEM1DT.ZIP, O3AMXEM1VY.ZIP.
    Files O3AMxEM2.* prepared for location table without EM22* location code sites: O3AMXEM2.LCT, O3AMXEM2.VOY, O3AMXEM2DT.ZIP, O3AMXEM2VY.ZIP, README.TXT
  • 6_VoyFixDataNo22SiteCode

  • Documentation:
  • Translation:, EMEFSLct.xls
  • VoyDat: EMEFSLST.DAD, EMEFSLST.DAH, EMEFSLST.LCD, EMEFSLST.LCT, EMEFSLST.VOY, EMEFSLST.VRD, EMEFSLST.VRT, EMEFSLSTDT.ZIP, EMEFSLSTVY.ZIP, README.TXT
  • 7_O3VoyFixDataNo22SiteCode

  • Documentation:
  • Translation:,
  • VoyDat: O3ELTN22.DAD, O3ELTN22.DAH, O3ELTN22.DAT, O3ELTN22.LCD, O3ELTN22.LCT, O3ELTN22.RPT, O3ELTN22.VOY, O3ELTN22.VRD, O3ELTN22.VRT, README.TXT



    SCION

    SCION is a research monitoring network operated in support of the Southern Oxidant Study (SOS). The SCION data supplied for OTAG include 10 monitoring sites over the southeastern US. The periods included were July, 1993 and July, 1995. Ozone was measured at all sites. At some sites data for CO, NO, and NOY were also available.

    1_DataSource

    The SCION data set was obtained from ESE Environmental, Inc. on diskettes. The data dictionaries required some clarification from the data supplier. The raw data were stored in the following files: CNVMETTR.ZIP, FORM_ACC.ZIP, METTRACE.ZIP, SCIONACC.LDB, SCIONACC.MDB, VOCROW.ZIP

    2_VoySource

  • Documentation:
  • Translation:,
  • VoyDat: OZ2.WKB, SCION.LDB, SCION.MDB, SCION.WKB, SCIONDT.ZIP, SCIONMET.DAD, SCIONMET.DAH, SCIONMET.LCD, SCIONMET.LCT, SCIONMET.VOY, SCIONMET.VRD, SCIONMET.VRT, SCIONVY.ZIP

    3_VoyFixData

  • Documentation:
  • Translation:,
  • VoyDat: SCIONFIX.LDB, SCIONFIX.MDB

    4_Voy03

  • Documentation:
  • Translation:,
  • VoyDat: O3SCDT.ZIP, O3SCION.DAD, O3SCION.DAH, O3SCION.LCD, O3SCION.LCT, O3SCION.VOY, O3SCION.VRD, O3SCION.VRT, O3SCVY.ZIP

    5_Voy03AgMx

  • Documentation:
  • Translation:,
  • VoyDat:

    Back to Contents


    LADCO

    The LADCO monitoring network is part of the Lake Area Ozone Research Program.

    The LADCO monitoring network is part of the Lake Area Ozone Research Program. The LADCO data supplied for OTAG model comparison consisted of 6 sites for the 1991 episode. For the monitoring site on the top of Sears Tower data were available for July of 1988, 91, 93, and 95.

    1_DataSource

    The ozone data for individual LADCO sites were transferred to the CAPITA FTP mailbox. The were no data dictionaries supplied, in fact four out of six sites had different data formats requiring clarification from LADCO. The clarification was rawLADCO.ZIP

    2_VoySource

  • Documentation:
  • Translation:,
  • VoyDat: CONV_DAT.ZIP, LADCO.DAD, LADCO.DAH, LADCO.LCD, LADCO.LCT, LADCO.LDB, LADCO.MDB, LADCO.VOY, LADCO.VRD, LADCO.VRT, LADCODT.ZIP, LADCOVY.ZIP, README.TXT

    3_VoyFixData

  • Documentation:
  • Translation:,
  • VoyDat: LADCOFIX.LDB, LADCOFIX.MDB, README.TXT

    4_Voy03

  • Documentation:
  • Translation:,
  • VoyDat: O3LADCO.DAD, O3LADCO.DAH, O3LADCO.LCD, O3LADCO.LCT, O3LADCO.VOY, O3LADCO.VRD, O3LADCO.VRT, O3LADDT.ZIP, O3LADVY.ZIP, README.TXT

    5_Voy03AgMx

  • Documentation:
  • Translation:,
  • VoyDat:

    Back to Contents


    GEORGIA SITES

    The special Georgia data are for 3 ozone monitoring sites covering the period 1987-1995.

    1_DataSource

    Three AIRS sites in Georgia could not be downloaded from AIRS because CAPITA did not have access privileges to these Georgia sites. The data were supplied from State of Georgia using the CAPITA FTP mailbox. There was no data dictionary, but the format was identical to one of the LADCO sites. The raw data were stored in:, gao3airs.zip, gao3airs.txt

    2_VoySource

  • Documentation:
  • Translation:, AIRS2DAT.BAS, AIRS2DAT.EXE, AIRS2DAT.FRM, AIRS2DAT.LOG, AIRS2DAT.VBP, CONVERT.TXT
  • VOYDAT: FORMAT.TXT, GAO3AIRS.VOY, GEORGDT.ZIP, GEORGVY.ZIP

    3_VoyFixData: README.TXT

  • Documentation:
  • Translation:,
  • VoyDat: GEORGFIX.LDB, GEORGFIX.MDB

    4_Voy03

  • Documentation:
  • Translation:,
  • VoyDat:

    VoyDat: GEORGIA.LDB, GEORGIA.MDB, O3GADT.ZIP, O3GAVY.ZIP, O3GEORG.DAD, O3GEORG.DAH, O3GEORG.LCD, O3GEORG.LCT, O3GEORG.VOY, O3GEORG.VRD, O3GEORG.VRT

    5_Voy03AgMx

  • Documentation:
  • Translation:,


  • Back to Contents

    NORTH CAROLINA SITES

    1_DataSource: OZONE.DAT

    2_VoySource

  • Documentation:
  • Translation:,
  • VoyDat: NC.LDB, NC.MDB, O3NCJL93.DAD, O3NCJL93.DAH, O3NCJL93.LCD, O3NCJL93.LCT, O3NCJL93.VOY, O3NCJL93.VRD, O3NCJL93.VRT, O3NCJL93DT.ZIP, O3NCJL93VY.ZIP

    3_VoyFixData

  • Documentation:
  • Translation:,
  • VoyDat: NCFIX.LDB, NCFIX.MDB

    4_Voy03

  • Documentation:
  • Translation:,
  • VoyDat: O3NCDT.ZIP, O3NCJL93.DAD, O3NCJL93.DAH, O3NCJL93.LCD, O3NCJL93.LCT, O3NCJL93.VOY, O3NCJL93.VRD, O3NCJL93.VRT, O3NCVY.ZIP

    5_Voy03AgMx

  • Documentation:
  • Translation:,
  • VoyDat:

    Back to Contents


    Submit your comments, feedback, questions, and ideas pertaining this page. Your input will be automatically added to the existing annotations. In order to add a new comment, you must be registered with the OTAG/AQA Peoples Page.