On Nonconvex Decentralized Gradient Descent

Jinshan Zeng Jiangxi Normal University Wotao Yin University of California, Los Angeles

Optimization and Control mathscidoc:1903.27002

IEEE Transactions on Signal Processing, 66, (11), 2018.6
Consensus optimization has received considerable attention in recent years. A number of decentralized algorithms have been proposed for convex consensus optimization. However, to the behaviors or consensus nonconvex optimization, our understanding is more limited. When we lose convexity, we cannot hope that our algorithms always return global solutions though they sometimes still do. Somewhat surprisingly, the decentralized consensus algorithms, DGD and Prox-DGD, retain most other properties that are known in the convex setting. In particular, when diminishing (or constant) step sizes are used, we can prove convergence to a (or a neighborhood of) consensus stationary solution under some regular assumptions. It is worth noting that Prox-DGD can handle nonconvex nonsmooth functions if their proximal operators can be computed. Such functions include SCAD, MCP, and lq quasinorms, q ∈ [0, 1). Similarly, Prox-DGD can take the constraint to a nonconvex set with an easy projection. To establish these properties, we have to introduce a completely different line of analysis, as well as modify existing proofs that were used in the convex setting.
Nonconvex dencentralized computing, consensus optimization, decentralized gradient descentmethod, proximal decentralized gradient descent
[ Download ] [ 2019-03-19 20:18:02 uploaded by JinshanZeng ] [ 770 downloads ] [ 0 comments ]
  title={On Nonconvex Decentralized Gradient Descent},
  author={Jinshan Zeng, and Wotao Yin},
  booktitle={IEEE Transactions on Signal Processing},
Jinshan Zeng, and Wotao Yin. On Nonconvex Decentralized Gradient Descent. 2018. Vol. 66. In IEEE Transactions on Signal Processing. http://archive.ymsc.tsinghua.edu.cn/pacm_paperurl/20190319201802846572205.
Please log in for comment!
Contact us: office-iccm@tsinghua.edu.cn | Copyright Reserved