A Data Management System DMS for Air Quality Management Agencies

Presentation Transcript

A Data Management System (DMS) for Air Quality Management Agencies Prepared by: Mark Stoelting Bay Area Air Quality Management District San Francisco, CA and Eric Gray Sonoma Technology, Inc. Petaluma, CA Presented at: U.S. EPA 2008 National Air Quality Conference Portland, OR April 6-9, 2008

DMS: Why, What, How, Who Why construct a DMS? What is it? How does DMS function? What are the DMS capacities? Who is included? Who paid for DMS? Where is it going?

Why the DMS? New innovation: instruments, correspondences, information get to Real-time information get to Improve information quality (QC checks) Graphical information exploring apparatuses Minimize/take out requirement for paper Track instruments and execution Shared need by all observing organizations

What is DMS? Information Management System Database to ingest, handle, trade information Archive Programmable database with discretionary components Process sub-hourly information (1 minute) Collect instrument metadata Track instruments and execution (OpCodes) Automatic Quality Checks Export (AirNow, AQS) Notification (email, prepare controls)

Simple Data Structure INSTRUMENT Site,Parameter,Date_Time,OpCode,Data,[alpha_data] 70 11 ,CO_chamt,10/05/2007 08:16,0,48.1580, (discretionary, doled out in database) {OpCode} SITE PARAMETER DATE_TIME DATA

Sites, Parameters, Instruments Establish Monitoring Sites Create Parameters Create Instruments

Assign Parameters to Instruments and Instruments to Sites Assign Parameters to Instruments Assign Instruments to Sites

Automated Quality Checks/Export Ambient Air Quality Data Raw Data Range checks Instrument checks AIRNow Site checks Network (off-website) checks District Web Page Exports separated by QC code

Automatic Quality Checks Assign Automatic Quality Control Checks to Parameters

Data Input Process Ingests information documents produced by datalogger or information organizing programming Duplicate info information is consequently dismisses Automated Quality Checks build up beginning QC code Min/max, zero float, staying, rate-of-progress, analyze QC conditions can actuate handle/caution Electronic chain-of-authority made Sub-hourly information amassed to 1-hr midpoints Automated information fare to AIRNow and Air District site

DMS Status Station Reporting Status Event Log QC Log

Manual Data Review Time-arrangement graphical altering and QC apparatuses for staff information audit DMS records all surveys and manual conformity to QC status and information values (electronic chain-of-care) Tracks manual information analyst status Lock last information preceding fare Exports last information for administrative accommodation to AQS or different offices

Time Series Edit/QC Screen

Scatter Plot Analysis

Other DMS Features Data get to security (client rights administration) Maintains instrument gear database Supports electronic instrument and webpage logs Daily instrument execution reports (utilizing OpCodes)

DMS Software Requirements DMS have PC: Windows Server 2000/2003 or XP/Vista Microsoft SQL Server 2005 (all releases) DMS Client Software: .NET 2.0 Framework Windows 2000, XP, Vista Future devices may require Java and ERSI arcGIS

DMS Hardware DMS Server: Minimum server necessities – Pentium 4 tablet Must support Microsoft Windows Hardware controlled by preparing needs Laptop forms a huge number of records every hour and store millions Server can handle a great many records for each hour and store billions DMS Client: Pentium III class PC or better Broadband association with DMS Server

DMS Performance Benchmarks AIRNow DMC DMS will keep running in August on a Quad Core Intel processor with 8 GB of RAM. The working framework will be Windows Server 2003 64-bit Edition • Currently, AIRNow DMC forms 12 million records a day for more than 5000 locales • The DMS framework can handle 22 million records for every hour or ½ billion a day (authentic information import) • The Bay Area DMS framework will prepare 1 million information records for each day

DMS Storage Benchmarks The database will develop by 100 MB for each one million records One billion records can be put away in 100 GB information DMS inquiry execution has been tried with ½ billion information records AIRNow DMC will test execution for over a billion records by fall

DMS Can Be … An instrument observing framework An information examination framework An information Quality Check and Review framework An information trade device (to AirNow, AQS) An information stockroom for chronicled information

Agency Deployment DMS created utilizing AIRNow innovation in 2004 DMS to supplant AIRNow innovation by fall 2008 DMS being used by BAAQMD, SCAQMD, and CARB by end of 2008 AIRNow International will be founded on DMS for 2009 Stable DMS dissemination bundle accessible for office use by mid 2009 Looking for 3 or 4 extra organization advancement accomplices in 2008

Development and Funding DMS was improvement and subsidized by offices with an enthusiasm for the outcome Contributors include: Bay Area AQMD (BAAQMD) EPA Region IX, 103 Grant EPA National Environmental Information Exchange Network (NEIEN) Challenge Grant South Coast AQMD (SCAQMD) EPA (AirNow and AirNow International) California Air Resources Board (CARB) soon New DMS highlights must be accessible to all offices New components must not decrease DMS usefulness to different offices

Future DMS Development Enhanced view/alter highlights Viewing apparatuses - unique mark plots Statistical devices - box plots, reports Support for toxics, PM channel information, Special Purpose Monitoring Expanded detailing abilities Instrument Control Charts GIS mapping capacity, web administrations, and NEIEN XML information trade hub

DMS Summary Simple, adaptable, financially savvy Fast, secure, and productive operation Comprehensive quality control devices Applicable to a wide assortment of information Free to open offices under a share-product utilize assention

Contact Information Mark Stoelting Bay Area Air Quality Management District mstoelting@baaqmd.gov (415) 749-4619 Eric Gray Sonoma Technology, Inc. eric@sonomatech.com (707) 665-9900