Diffusion Mechanism in Residual Neural Network: Theory and Applications

Tangjun Wang epartment of Mathematical Sciences, Tsinghua University, Beijing 100084, China Zehao Dou Department of Statistics and Data Science, Yale University Chenglong Bao Yau Mathematical Sciences Center, Tsinghua University, Beijing 100084, China, and also with Yanqi Lake Beijing Institute of Mathematical Sciences and Applications, Beijing 101408, Chin Zuoqiang Shi au Mathematical Sciences Center, Tsinghua University, Beijing 100084, China, and also with Yanqi Lake Beijing Institute of Mathematical Sciences and Applications, Beijing 101408, China

Machine Learning mathscidoc:2206.41002

2022.5
Diffusion, a fundamental internal mechanism emerging in many physical processes, describes the interaction among different objects. In many learning tasks with limited training samples, the diffusion connects the labeled and unlabeled data points and is a critical component for achieving high classification accuracy. Many existing deep learning approaches directly impose the fusion loss when training neural networks. In this work, inspired by the convection-diffusion ordinary differential equations (ODEs), we propose a novel diffusion residual network (Diff-ResNet), internally introduces diffusion into the architectures of neural networks. Under the structured data assumption, it is proved that the proposed diffusion block can increase the distance-diameter ratio that improves the separability of inter-class points and reduces the distance among local intra-class points. Moreover, this property can be easily adopted by the residual networks for constructing the separable hyperplanes. Extensive experiments of synthetic binary classification, semi-supervised graph node classification and few-shot image classification in various datasets validate the effectiveness of the proposed method.
No keywords uploaded!
[ Download ] [ 2022-06-13 16:53:03 uploaded by Baocl ] [ 59 downloads ] [ 0 comments ]
@inproceedings{tangjun2022diffusion,
  title={Diffusion Mechanism in Residual Neural Network: Theory and Applications},
  author={Tangjun Wang, Zehao Dou, Chenglong Bao, and Zuoqiang Shi},
  url={http://archive.ymsc.tsinghua.edu.cn/pacm_paperurl/20220613165303929586348},
  year={2022},
}
Tangjun Wang, Zehao Dou, Chenglong Bao, and Zuoqiang Shi. Diffusion Mechanism in Residual Neural Network: Theory and Applications. 2022. http://archive.ymsc.tsinghua.edu.cn/pacm_paperurl/20220613165303929586348.
Please log in for comment!
 
 
Contact us: office-iccm@tsinghua.edu.cn | Copyright Reserved