MathSciDoc: An Archive for Mathematician ∫

 
Log In
Sign Up
Help
Go
 

Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detection

Penghang Yin University of California, Los Angeles Shuai Zhang University of California, Irvine Yingyong Qi University of California, Irvine Jack Xin University of California, Irvine

Information Theory mathscidoc:1802.19005

arXiv, 2017
We present LBW-Net, an efficient optimization based method for quantization and training of the low bit-width convolutional neural networks (CNNs). Specifically, we quantize the weights to zero or powers of two by minimizing the Euclidean distance between full-precision weights and quantized weights during backpropagation. We characterize the combinatorial nature of the low bit-width quantization problem. For 2-bit (ternary) CNNs, the quantization of N weights can be done by an exact formula in O(N log N ) complexity. When the bit-width is three and above, we further propose a semi-analytical thresholding scheme with a single free parameter for quantization that is computationally inexpensive. The free parameter is further determined by network retraining and object detection tests. LBW-Net has several desirable advantages over full-precision CNNs, including considerable memory savings, energy efficiency, and faster deployment. Our experiments on PASCAL VOC dataset show that compared with its 32-bit floating-point counterpart, the performance of the 6-bit LBW-Net is nearly lossless in the object detection tasks, and can even do better in some real world visual scenes, while empirically enjoying more than 4× faster deployment.
No keywords uploaded!
[ Download ] [ 2018-02-14 11:35:58 uploaded by jack ] [ 993 downloads ] [ 0 comments ] 
@inproceedings{penghang2017quantization,
  title={Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detection},
  author={Penghang Yin, Shuai Zhang, Yingyong Qi, and Jack Xin},
  url={http://archive.ymsc.tsinghua.edu.cn/pacm_paperurl/20180214113558930978929},
  booktitle={arXiv},
  year={2017},
}
Penghang Yin, Shuai Zhang, Yingyong Qi, and Jack Xin. Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detection. 2017. In arXiv. http://archive.ymsc.tsinghua.edu.cn/pacm_paperurl/20180214113558930978929.
Please log in for comment!
 
 
Contact us: office-iccm@tsinghua.edu.cn | Copyright Reserved