Files in this item



application/pdfHUANG-DISSERTATION-2016.pdf (42MB)
(no description provided)PDF


Title:Visual analysis and synthesis with physically grounded constraints
Author(s):Huang, Jia-Bin
Director of Research:Ahuja, Narendra
Doctoral Committee Chair(s):Ahuja, Narendra
Doctoral Committee Member(s):Huang, Thomas S.; Do, Minh N.; Hasegawa-Johnson, Mark; Hoiem, Derek
Department / Program:Electrical & Computer Eng
Discipline:Electrical & Computer Engr
Degree Granting Institution:University of Illinois at Urbana-Champaign
Subject(s):Computer vision
Visual synthesis
patch-based optimization
image completion
image super-resolution
video completion
visual tracking
Abstract:The past decade has witnessed remarkable progress in image-based, data-driven vision and graphics. However, existing approaches often treat the images as pure 2D signals and not as a 2D projection of the physical 3D world. As a result, a lot of training examples are required to cover sufficiently diverse appearances and inevitably suffer from limited generalization capability. In this thesis, I propose "inference-by-composition" approaches to overcome these limitations by modeling and interpreting visual signals in terms of physical surface, object, and scene. I show how we can incorporate physically grounded constraints such as scene-specific geometry in a non-parametric optimization framework for (1) revealing the missing parts of an image due to removal of a foreground or background element, (2) recovering high spatial frequency details that are not resolvable in low-resolution observations. I then extend the framework from 2D images to handle spatio-temporal visual data (videos). I demonstrate that we can convincingly fill spatio-temporal holes in a temporally coherent fashion by jointly reconstructing the appearance and motion. Compared to existing approaches, our technique can synthesize physically plausible contents even in challenging videos. For visual analysis, I apply stereo camera constraints for discovering multiple approximately linear structures in extremely noisy videos with an ecological application to bird migration monitoring at night. The resulting algorithms are simple and intuitive while achieving state-of-the-art performance without the need of training on an exhaustive set of visual examples.
Issue Date:2016-07-06
Rights Information:Copyright 2016 Jia-Bin Huang
Date Available in IDEALS:2016-11-10
Date Deposited:2016-08

This item appears in the following Collection(s)

Item Statistics