No abstract uploaded!

No abstract uploaded!

In this paper, we develop an alternating direction method of multipliers (ADMM) for deep neural networks training with sigmoid-type activation functions (called sigmoid-ADMM pair ), mainly motivated by the gradient-free nature of ADMM in avoiding the saturation of sigmoid-type activations and the advantages of deep neural networks with sigmoid-type activations (called deep sigmoid nets) over their rectified linear unit (ReLU) counterparts (called deep ReLU nets) in terms of approximation. In particular, we prove that the approximation capability of deep sigmoid nets is not worse than that of deep ReLU nets by showing that ReLU activation function can be well approximated by deep sigmoid nets with two hidden layers and finitely many free parameters but not vice-verse. We also establish the global convergence of the proposed ADMM for the nonlinearly constrained formulation of the deep sigmoid nets training from arbitrary initial points to a Karush-Kuhn-Tucker (KKT) point at a rate of order O(1/k). Besides sigmoid activation, such a convergence theorem holds for a general class of smooth activations. Compared with the widely used stochastic gradient descent (SGD) algorithm for the deep ReLU nets training (called ReLU-SGD pair), the proposed sigmoid-ADMM pair is practically stable with respect to the algorithmic hyperparameters including the learning rate, initial schemes and the pro-processing of the input data. Moreover, we find that to approximate and learn simple but important functions the proposed sigmoid-ADMM pair numerically outperforms the ReLU-SGD pair.

Shape registration has a wide range of applications in geometric modeling, medical imaging, and computer vision. This paper focuses on the registration of the genus-3 vestibular systems and studies the geometric differences between the normal and Adolescent Idiopathic Scoliosis (AIS) groups. The non-trivial topology of the VS poses great technical challenges to the geometric analysis. To tackle these challenges, we present an effective and practical solution to register the vestibular systems. We first extract six geodesic landmarks for the VS, which are stable, intrinsic, and insensitive to the VS’s resolution and tessellation. Moreover, they are highly consistent regardless of the AIS and normal groups. The detected geodesic landmarks partition the VS into three patches, a topological annulus and two topological disks. For each pair of patches of the AIS subject and the control, we compute a bijective map using the holomorphic 1-form and harmonic map techniques. With a carefully designed boundary condition, the three individual maps can be glued in a seamless manner so that the resulting registration is a homeomorphism with exact landmark matching. Our method is robust, automatic and efficient. It takes only a few seconds on a low-end PC, which significantly outperforms the non-rigid ICP algorithm. We conducted a student’s t-test on the test data. Computational results show that using the mean curvature measure E_H, our method can distinguish the AIS subjects and the normal subjects.

No abstract uploaded!