Conformal prediction with temporal updates in cross-sectional time series

Lin, Zhen

Permalink

https://hdl.handle.net/2142/120346 Copy

Description

Title

Conformal prediction with temporal updates in cross-sectional time series

Author(s)

Lin, Zhen

Issue Date

2023-04-04

Director of Research (if dissertation) or Advisor (if thesis)

Sun, Jimeng

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Conformal Prediction
• Cross-sectional Time Series
• Healthcare Applications

Language

eng

Abstract

Cross-sectional time series data is common in several important domains, especially healthcare. A classical example in healthcare is predicting patient outcomes or certain biomarkers using electronic health records: A population of patients forms a cross-section, and each patient can be represented by one time series. An important task in such prediction setting is constructing reliable prediction intervals (PIs) that are valid. That is, the PIs should cover the true response value with a high probability (such as 90%) chosen in advance. PI estimators in this setting, however, should address two different notions of coverage - cross-sectional coverage and longitudinal coverage. The former could be considered a probability measured across a cross-sectional slice, and the latter along the temporal dimension for each time series. Recent works started to adapt Conformal Prediction (CP) to obtain PIs for time series forecasting, but works that handle both notions of coverage simultaneously are still missing. In this work, we first identify and distinguish the two types of validity. Then, we show that unlike cross-sectional validity, strict longitudinal validity is impossible to achieve in a distribution-free manner with finite-samples. We then proceed to propose methods that enjoy cross-sectional validity and can also improved longitudinal coverage. The first class of methods, called Conformal Prediction with Temporal Dependence, or CPTD, uses temporally and cross-sectionally informed nonconformity scores to adapt to the information observed so far. The second class of methods, called Temporal Quantile Adjustments, or TQA, adjusts the quantile queried by CP methods at each time step t. Both methods account for both cross-sectional and longitudinal coverage with strong theoretical guarantees. Moreover, both CPTD and TQA could be used as a general wrapper for any time series regression model, thanks for their post-hoc nature. We validate their performance through extensive experiments, and show that both CPTD and TQA generally obtain efficient PIs and improve longitudinal coverage while preserving cross-sectional coverage.

Graduation Semester

2023-05

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/120346

Copyright and License Information

Copyright 2023 Zhen Lin

Owning Collections