In this paper, we develop an alternating direction method of multipliers (ADMM) for deep neural networks training with sigmoid-type activation functions (called sigmoid-ADMM pair ), mainly motivated by the gradient-free nature of ADMM in avoiding the saturation of sigmoid-type activations and the advantages of deep neural networks with sigmoid-type activations (called deep sigmoid nets) over their rectified linear unit (ReLU) counterparts (called deep ReLU nets) in terms of approximation. In particular, we prove that the approximation capability of deep sigmoid nets is not worse than that of deep ReLU nets by showing that ReLU activation function can be well approximated by deep sigmoid nets with two hidden layers and finitely many free parameters but not vice-verse. We also establish the global convergence of the proposed ADMM for the nonlinearly constrained formulation of the deep sigmoid nets training from arbitrary initial points to a Karush-Kuhn-Tucker (KKT) point at a rate of order O(1/k). Besides sigmoid activation, such a convergence theorem holds for a general class of smooth activations. Compared with the widely used stochastic gradient descent (SGD) algorithm for the deep ReLU nets training (called ReLU-SGD pair), the proposed sigmoid-ADMM pair is practically stable with respect to the algorithmic hyperparameters including the learning rate, initial schemes and the pro-processing of the input data. Moreover, we find that to approximate and learn simple but important functions the proposed sigmoid-ADMM pair numerically outperforms the ReLU-SGD pair.

We apply a study of orders in quaternion algebras, to the differential geometry of Riemann surfaces. The least length of a closed geodesic on a hyperbolic surface is called its systole,and denoted sysπ1. P. Buser and P. Sarnak constructed Riemann surfaces X whose systole behaves logarithmically in the genus g(X). The Fuchsian groups in their examples are principal congruence subgroups of a fixed arithmetic group with rational trace field. We generalize their construction to principal congruence subgroups of arbitrary arithmetic surfaces. The key tool is a new trace estimate valid for an arbitrary ideal in a quaternion algebra. We obtain a particularly sharp bound for a principal congruence tower of Hurwitz surfaces (PCH), namely the 4/3bound sysπ1(XPCH) ≥ 43 log(g(XPCH)). Similar results are obtained for the systole of hyperbolic 3-manifolds, relative to their simplicial volume.

In this paper, we develop a three-dimensional parallel solver using the fifth order high-resolution weighted essentially non-oscillatory (WENO) finite difference scheme to perform extensive simulation for three-dimensional gaseousdetonations. A careful study is conducted for the propagation modes of three-dimensional gaseous detonation wave-front structures in a long square duct. The numerical results indicate that, the instability of detonation, overdrive factor, transverse dimension and initial perturbation are the main factors that influence the appearance of spinning detonation.

No abstract uploaded!

No abstract uploaded!