Learning to super-resolve images using self-similarities

Abstract

The single image super-resolution problem entails estimating a high-resolution version of a low-resolution image. Recent studies have shown that high resolution versions of the patches of a given low-resolution image are likely to be found within the given image itself. This recurrence of patches across scales in an image forms the basis of self-similarity driven algorithms for image super-resolution. Self-similarity driven approaches have the appeal that they do not require any external training set; the mapping from low-resolution to high-resolution is obtained using the cross scale patch recurrence. In this dissertation, we address three important problems in super-resolution, and present novel self-similarity based solutions to them: First, we push the state-of-the-art in terms of super-resolution of fine textural details in the scene. We propose two algorithms that use self-similarity in conjunction with the fact that textures are better characterized by their responses to a set of spatially localized bandpass filters, as compared to intensity values directly. Our proposed algorithms seek self-similarities in the sub-bands of the image, for better synthesizing fine textural details. Second, we address the problem of super-resolving an image in the presence of noise. To this end, we propose the first super-resolution algorithm based on self-similarity that effectively exploits the high-frequency content present in noise (which is ordinarily discarded by denoising algorithms) for synthesizing useful textures in high-resolution. Third, we present an algorithm that is able to better super-resolve images containing geometric regularities such as in urban scenes, cityscapes etc. We do so by extracting planar surfaces and their parameters (mid-level cues) from the scene and exploiting the detected scene geometry for better guiding the self-similarity search process. Apart from the above self-similarity algorithms, this dissertation also presents a novel edge-based super-resolution algorithm that super-resolves an image by learning from training data how edge profiles transform across resolutions. We obtain edge profiles via a detailed and explicit examination of local image structure, which we show to be more robust and accurate as compared to conventional gradient profiles.