Using information theoretic measures to evaluate support vector machine kernels

Abstract

A new method is proposed that exploits the underlying information theoretic structure in the input data to evaluate the ability of a kernel to successfully separate a class in some feature space. This method is built on the fundamental idea that kernel density estimation in some input space is equivalent to an inner product on some Hilbert space. Estimators of Renyi's generalized form of information theoretic measurements reduce to a form that gives an elegant characterization of the geometric properties of the kernel in the feature space. It is shown how these estimators can be used to evaluate the kernel of a support vector machine.