Abstract—This study proposes a collaborative approach between algorithms and hardware to enhance the efficiency and effectiveness of deep learning algorithms by exploring hardware architectures most suitable for efficiently executing deep learning algorithms. This co-design approach comprises algorithms and hardware to optimize model complexity and build resource-efficient solutions. Regularization techniques are employed to reduce computational demands from deep learning models to enhance the effectiveness and accuracy of deep learning outputs in deep compression applications. Additionally, this research also develops an Efficient Inference Engine (EIE), a specialized hardware accelerator designed for direct inference on compressed models, significantly improving performance and energy efficiency. The techniques utilized in this research encompass Deep Compression (DC), Dense-Sparse-Dense (DSD), and EIE. The DSD technique involves periodic pruning and restoration of connections in deep learning models to enhance prediction accuracy and prevent overfitting. For the EIE technique, direct inference is performed on compressed models, thus saving memory bandwidth and improving inference speed and energy efficiency. The results of this research show that the development of DC techniques, DSD training, and the EIE hardware architecture have successfully improved deep learning efficiency. With the deep compression model, DNN models can be compressed by 17× up to 49× without sacrificing prediction accuracy. Meanwhile, DSD improves prediction accuracy for various deep learning models (convolutional neural networks, recurrent neural networks, and long short term memory networks). Lastly, EIE has achieved a speed improvement of 13-fold and energy efficiency of 3,400-fold compared to GPUs, enabling faster, more energy-efficient, and more accurate usage in various applications.
 
Keywords—Deep Compression (DC), Efficient Inference Engine (EIE), Dense-Sparse-Dense (DSD), algorithm, hardware co-design


Copyright © 2024 by the authors. This is an open access article distributed under the Creative Commons Attribution License (CC BY-NC-ND 4.0), which permits use, distribution and reproduction in any medium, provided that the article is properly cited, the use is non-commercial and no modifications or adaptations are made.