Bayesian high dimensional modeling with group structures

Abstract

Group structures arise naturally in a variety of modern data applications and statistical problems in the high-dimensional data setting where the number of variables can greatly exceed the number of observations. The group structures are usually very informative as they express the inherent similarities among the variables and observations and it is thus desirable to take the prior group information into consideration in the construction of statistical models in pursuit of efficient statistical inference. In this dissertation, we propose methods for three statistical problems: linear regression, graphical model, and sequential logistic regressions when group structures are present at either the variable level or the observation level. We adopt the Bayesian framework and extend the spike-and-slab priors in the group setting to incorporate the group information. Our proposed hierarchical Bayesian models are well suited for sharing of similar sparsity patterns within the groups. For posterior computation, we propose EM algorithms and shotgun stochastic search algorithm which are more efficient than standard Markov chain Monte Carlo sampling algorithms. Compared to methods that simply ignore the grouping structure, our proposed methods that involve the group information can lead to better statistical inference results and the fitted models are more interpretable and provide more insights into the data. Further, we show that our methods are also advantageous theoretically or empirically in comparison with other group selection competitors due to the nonconvex regularization induced from our Bayesian modeling.