Files in this item



application/pdfQIAN-THESIS-2020.pdf (4MB)Restricted Access
(no description provided)PDF


Title:Profiling and characterization of deep learning model inference on CPU
Author(s):Qian, Yanli
Advisor(s):Hwu, Wen-Mei
Department / Program:Electrical & Computer Eng
Discipline:Electrical & Computer Engr
Degree Granting Institution:University of Illinois at Urbana-Champaign
Subject(s):Deep learning
Machine learning
Abstract:With the rapid growth of deep learning models and higher expectations for their accuracy and throughput in real-world applications, the demand for profiling and characterizing model inference on different hardware/software stacks is significantly increased. As the model inference characterization on GPU has already been extensively studied, it is worth exploring how performance-enhancing libraries like Intel MKL-DNN help to boost the performance on Intel CPU. We develop a profiling mechanism to capture the MKL-DNN operation calls and formulate the tracing timeline with spans on the server. Through profiling and characterization that give insights into Intel MKL-DNN, we evaluate and demonstrate that the optimization techniques, including blocked memory layout, layers fusion, and low precision operation used in deep learning model inference, have accelerated the performance on the Intel CPU.
Issue Date:2020-04-28
Rights Information:Copyright 2020 Yanli Qian
Date Available in IDEALS:2020-08-27
Date Deposited:2020-05

This item appears in the following Collection(s)

Item Statistics