Exploring structural regularities for robust 3D reconstruction of urban scenes

Zhou, Zihan

Permalink

https://hdl.handle.net/2142/44311 Copy

Description

Title

Exploring structural regularities for robust 3D reconstruction of urban scenes

Author(s)

Zhou, Zihan

Issue Date

2013-05-24T22:07:21Z

Director of Research (if dissertation) or Advisor (if thesis)

Ma, Yi

Doctoral Committee Chair(s)

Ma, Yi

Committee Member(s)

• Huang, Thomas S.
• Liang, Zhi-Pei
• Hoiem, Derek W.

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• 3D reconstruction
• structure from motion
• urban scene
• structural regularity

Abstract

Driven by emergent needs in industrial applications such as film production, navigation and virtual reality, the problem of inferring 3D structures of urban scenes from 2D images has recently drawn a lot of interest in the computer vision community. Despite the extremely rich literature in multiple view geometry and structure from motion (SFM), reconstructing large-scale high-quality 3D urban models still remains a challenging problem. A key feature of urban scene that differentiates it from other kinds of landscapes is the presence of strong structural regularities, such as planar surfaces, repetitive structures and all types of symmetries. While such regularities are largely ignored by existing SFM systems, in this thesis we demonstrate how they can be used to greatly facilitate 3D urban reconstruction as well as other related vision tasks. In the first part of the thesis, we first look into the problem of structure and motion recovery directly from one or more large planes in the scene. We develop a new SFM method that generates high-quality reconstruction results in a short time, while avoiding several practical difficulties of conventional methods. Then, we show how the recovered planar structures can be seamlessly integrated into the current state-of-the-art video stabilization systems to obtain high-quality jitter-free videos in many challenging cases. In the second part of the thesis, we focus on the structural regularities in visual data which give rise to a low-rank matrix structure, and develop a series of tools to recover them from images and videos. After reviewing the recent developments of convex optimization techniques for low-rank matrix recovery, we propose a novel 3D reconstruction approach based on a new class of global features called transform invariant low-rank textures (TILT). We demonstrate the advantage of such global features over traditional local features in handling large-baseline images, occlusions, and repetitive patterns. In addition, we extend the tools from low-rank matrix recovery to harness the redundancy and temporal correlations among a large number of video frames, which leads to a novel method for generating clean textured models for street views. For future work, my focus is on developing new methods for discovering complex structural regularities in urban scenes from large-scale visual data. I believe such methods would have a big impact in many modern industry applications in the near future.

Graduation Semester

2013-05

Permalink

http://hdl.handle.net/2142/44311

Copyright and License Information

Copyright 2013 Zihan Zhou

Owning Collections