Data Mining Approches to Complex Environmental Problems

Abstract

Anomaly detection is the task of identifying data that deviate from historical patterns. It has many practical applications, such as data quality assurance and control (QA/QC), focused data collection, and event detection. The second portion of this dissertation develops a suite of data-driven anomaly detection methods, based on autoregressive data-driven models (e.g. artificial neural networks) and dynamic Bayesian network (DBN) models of the sensor data stream. All of the developed methods perform fast, incremental evaluation of data as it becomes available; scale to large quantities of data; and require no a priori information, regarding process variables or types of anomalies that may be encountered. Furthermore, the methods can be easily deployed on large heterogeneous sensor networks. The anomaly detection methods are then applied to a sensor network located in Corpus Christi Bay, Texas, and their abilities to identify both real and synthetic anomalies in meteorological data are compared. Results of these case studies indicate that DBN-based detectors, using either robust Kalman filtering or Rao-Blackwellized particle filtering, are most suitable for the Corpus Christi meteorological data.