Files in this item



application/pdfTSOI-THESIS-2016.pdf (4MB)
(no description provided)PDF


Title:Improve OpenMVG and create a novel algorithm for novel view synthesis from point clouds
Author(s):Tsoi, Ka Wai
Advisor(s):Hoiem, Derek W.
Department / Program:Computer Science
Discipline:Computer Science
Degree Granting Institution:University of Illinois at Urbana-Champaign
Subject(s):Computer Vision
Computer Science
Novel View Synthesis
Abstract:This thesis presents work to improve open source 3D reconstruction software OpenMVG and to create a novel algorithm to render photorealistic images from new views given a photo collection and 3D point cloud. First, the original OpenMVG is parallelized using GPU and its data structure is optimized. Moreover, we integrated the MatchMiner algorithm into OpenMVG to further improve its efficiency. Last but not least, an initial pair selection formulation and a default focal length setting are introduced and implemented to automize OpenMVG. Then 3D sparse point clouds of construction sites are reconstructed by performing Structure-from-Motion (SfM) with the improved version of OpenMVG and source images (images that are used in SfM) are calibrated and registered to point clouds. Furukawa's Patch-based Multi-view Stereo(PMVS) algorithm is used to reconstruct dense point clouds using calibrated cameras as inputs. With known depth values of 3D points in the dense point cloud, we estimate depth maps of source images using optimization similar to Levin's colorization algorithm. For a novel view of the point cloud, we find source images that share some common elements of the construction site that are also visible to the novel view. Then we warp depth maps of these candidate images to the novel view. We estimate a depth map and label pixels for the novel view by solving a multi-label Markov Random Field (MRF) optimization problem using graph-cuts. We introduce a novel energy minimization formulation exploits both 2D and 3D information. Finally, a photorealistic image of the novel view is rendered by copying pixel colors from selected candidate source images using pixel labels computed with graph-cuts. We experimentally validate our approach on several challenging viewing angles of a point cloud model of a complicate construction site. The rendered results show high photo-realistic synthesis quality in planar scenes.
Issue Date:2016-04-26
Rights Information:2016 Ka Wai Tsoi.
Date Available in IDEALS:2016-07-07
Date Deposited:2016-05

This item appears in the following Collection(s)

Item Statistics