In plain words: It represents numbers as random streams of 0s and 1s so a neural network's hardware can be tiny, parallel, and low-power. An automatic allocator then picks every design setting together to fit a fixed budget while keeping accuracy.
Abstract · Towards Budget-Driven Hardware Optimization for Deep Convolutional Neural Networks using Stochastic Computing
Recently, Deep Convolutional Neural Network (DCNN) has achieved tremendous success in many machine learning applications. Nevertheless, the deep structure has brought significant increases in computation complexity. Largescale deep learning systems mainly operate in high-performance server clusters, thus restricting the application extensions to personal or mobile devices. Previous works on GPU and/or FPGA acceleration for DCNNs show increasing speedup, but ignore other constraints, such as area, power, and energy. Stochastic Computing (SC), as a unique data representation and processing technique, has the potential to enable the design of fully parallel and scalable hardware implementations of large-scale deep learning systems. This paper proposed an automatic design allocation algorithm driven by budget requirement considering overall accuracy performance. This systematic method enables the automatic design of a DCNN where all design parameters are jointly optimized. Experimental results demonstrate that proposed algorithm can achieve a joint optimization of all design parameters given the comprehensive budget of a DCNN.
Zhe Li, Ji Li, Ao Ren, Caiwen Ding, Jeffrey Draper, Qinru Qiu, Bo Yuan, Yanzhi Wang
arXiv:1805.04142 · cs.NE, cs.ET · submitted May 10, 2018
abstract · pdf · html · Accepted by IEEE Computer Society Annual Symposium on VLSI 2018