Learning efficient temporal information in deep networks: From the viewpoints of applications and modeling

Mac, Cu Khoi-Nguyen

Permalink

https://hdl.handle.net/2142/113820 Copy

Description

Title

Learning efficient temporal information in deep networks: From the viewpoints of applications and modeling

Author(s)

Mac, Cu Khoi-Nguyen

Issue Date

2021-10-13

Director of Research (if dissertation) or Advisor (if thesis)

Do, Minh N

Doctoral Committee Chair(s)

Do, Minh N

Committee Member(s)

• Forsyth, David A
• Hasegawa-Johnson, Mark A
• Schwing, Alexander G
• Gupta, Saurabh

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Deep learning
• Temporal information
• Efficiency
• Bandwidth extension
• Action recognition
• Action detection
• Sampling
• Optical flow

Abstract

With the introduction of deep learning, machine learning has dominated several technology areas, giving birth to high-performance applications that can even challenge human-level accuracy. However, the complexity of deep models is also exploding as a by-product of the revolution of machine learning. Such enormous model complexity has raised the new challenge of improving the efficiency in deep models to reduce deployment expense, especially for systems with high throughput demands or devices with limited power. The dissertation aims to improve the efficiency of temporal-sensitive deep models in four different directions. First, we develop a bandwidth extension mapping to avoid deploying multiple speech recognition systems corresponding to wideband and narrowband signals. Second, we apply a multi-modality approach to compensate for the performance of an excitement scoring system, where the input video sequences are aggressively down-sampled to reduce throughput. Third, we formulate the motion feature in the feature space by directly inducing the temporal information from intermediate layers of deep networks instead of relying on an additional optical flow stream. Finally, we model a spatiotemporal sampling network inspired by the human visual perception mechanism to reduce input frames and regions adaptively.

Graduation Semester

2021-12

Type of Resource

Thesis

Permalink

http://hdl.handle.net/2142/113820

Copyright and License Information

Copyright 2021 Cu Khoi Nguyen Mac

Owning Collections