Statistical error compensation for robust digital signal processing and machine learning

Kim, Eric

Permalink

https://hdl.handle.net/2142/72955 Copy

Description

Title

Statistical error compensation for robust digital signal processing and machine learning

Author(s)

Kim, Eric

Issue Date

2015-01-21

Director of Research (if dissertation) or Advisor (if thesis)

Shanbhag, Naresh R.

Doctoral Committee Chair(s)

Shanbhag, Naresh R.

Committee Member(s)

• Rutenbar, Rob
• Singer, Andrew C.
• Chen, Deming

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• statistical error compensation
• low power design
• error resiliency
• machine learning
• digital signal processing

Abstract

Machine learning (ML) based inference has recently gained importance as a key kernel in processing massive data in digital signal processing (DSP) systems. Due to the ever increasing complexity of DSP systems, energy-efficient ML accelerators are critical. Traditionally, energy efficiency was obtained through technology scaling. However, modern nanoscale complementary metal–oxide semiconductor (CMOS) process technologies suffer in reliability caused by process, temperature, and voltage variations. As ML applications are inherently probabilistic and robust to errors, statistical error compensation (SEC) techniques can play a significant role in achieving robust and energy-efficient implementation of these important kernels. SEC embraces the statistical nature of errors and utilizes statistical and probabilistic techniques to build robust systems. Energy efficiency is obtained by trading off the enhanced robustness with energy. This dissertation focuses on utilizing statistical approaches via SEC in implementing energy-efficient digital signal processing (DSP) systems with an emphasis on machine learning kernels. We first demonstrate the potential of SEC techniques to a detection based application. A 180nm CMOS pseudonoise (PN) code acquisition integrated circuit (IC) has been implemented and measured. Measurements show that while maintaining a detection probability Pdet >= 90%, an error rate p_eta >= 85.83% with energy savings of 2.52X could be achieved. SEC is then applied to a communication centric machine learning kernel, a low-density parity check (LDPC) decoder. As iterative message-passing based architectures are inherently robust to small-magnitude errors, the SEC based LDPC decoder shows significant improvement in robustness and energy efficiency. Three different size LDPC codes, (50, 25), (800, 400), and (1800, 900), were implemented with five iterations per block. Circuit simulations in a commercial 45nm process show that the SEC based LDPC decoder can operate at a supply voltage up to 38% less than the nominal voltage and tolerate up to 30X more errors over an SNR range of 3 dB to 8 dB, while maintaining less than 3X degradation in bit error rate (BER). This is equivalent to energy savings of 45.7% compared to conventional LDPC decoders, and 33.2% compared to a sign bit protected LDPC decoder. Motivated by the success of SEC based LDPC decoders, SEC has been applied to a more complex message-passing application: Markov random field (MRF) based stereo image matching. Analysis and simulations show that for a 20-bit architecture, small errors (eta <= 1024) are tolerable, while large errors (eta >= 4096) degrade the performance significantly. By applying algorithmic noise tolerance (ANT), experimental results show that the proposed ANT based hardware can tolerate an error rate of 20%, with performance degradation of only 3.5% at an overhead of 97.4%, compared to an error-free full precision hardware with an energy savings of 39.6%. To reduce the compensation complexity, higher level error compensation is explored as well. Recent studies on approximate computing (AC) follow a principle similar to SEC, but with one critical exception. AC based design still carries the requirement of creating a deterministic design, and thus the improvement in energy efficiency is marginal. We successfully apply SEC to AC based designs and show that by embracing the statistical nature of the underlying process, an additional 44.9% energy savings can be obtained. Finally, SEC techniques are analyzed to provide insight into the trade-offs in the design of SEC based systems. Algorithmic noise tolerance is analyzed under a unifying framework based on detection and estimation theory. ANT is shown to approximate the Bayes optimal detector and estimator.

Graduation Semester

2014-12

Permalink

http://hdl.handle.net/2142/72955

Copyright and License Information

Copyright 2014 Eric Kim

Owning Collections