Files in this item



application/pdfFA20-ECE499-Thesis-Sinclair, Finn.pdf (5MB)
(no description provided)PDF


Title:Spatio-temporal reprojection for virtual and augmented reality applications
Author(s):Finn, Sinclair
Contributor(s):Adve, Sarita
virtual reality
augmented reality
Abstract:Late-stage reprojection is a vital tool for reducing power and compute requirements and improving perceptual latency in extended reality (XR) systems. While rotational reprojection is a well-studied topic for which several implementations have been published, the more advanced type of reprojection, spatial reprojection, is largely uncharacterized and unpublished. Depth-aware spatio-temporal reprojection can reproject a depth volume to account for the full six-degrees-of- freedom of user motion. By reprojecting application frames with the full range of user motion with a sufficiently high level of accuracy, dropped frames and display latency can potentially be almost completely resolved. In addition, with the advent of remote-rendering and edge-compute-based architectures, the reprojection of rendered application frames can mitigate latencies introduced by the transmission of frames across great distances. This thesis aggregates and reviews prior related work in the domain of 3D image-based reprojection and warping algorithms, examining the applicability of previous reprojection algorithms to XR applications. We evaluate three different methods for solving the warp equation, weighing their characteristics, advantages, and disadvantages across several representative XR scenarios. Finally, the thesis presents two novel implementations of full depth-aware spatio-temporal reprojection, as well as deep performance characterizations on multiple representative platforms. In particular, the algorithms’ resilience to occlusion artifacts, high-frequency depth features, and large perspective displacements are examined and characterized. This paper also presents a detailed analysis of the structural similarity index measure (SSIM) of reprojected frames to characterize the quality of the algorithms’ output and the contribution to the overall user experience.
Issue Date:2020-12
Date Available in IDEALS:2021-01-09

This item appears in the following Collection(s)

Item Statistics