In this paper, we develop an alternating direction method of multipliers (ADMM) for deep neural networks training with sigmoid-type activation functions (called sigmoid-ADMM pair ), mainly motivated by the gradient-free nature of ADMM in avoiding the saturation of sigmoid-type activations and the advantages of deep neural networks with sigmoid-type activations (called deep sigmoid nets) over their rectified linear unit (ReLU) counterparts (called deep ReLU nets) in terms of approximation. In particular, we prove that the approximation capability of deep sigmoid nets is not worse than that of deep ReLU nets by showing that ReLU activation function can be well approximated by deep sigmoid nets with two hidden layers and finitely many free parameters but not vice-verse. We also establish the global convergence of the proposed ADMM for the nonlinearly constrained formulation of the deep sigmoid nets training from arbitrary initial points to a Karush-Kuhn-Tucker (KKT) point at a rate of order O(1/k). Besides sigmoid activation, such a convergence theorem holds for a general class of smooth activations. Compared with the widely used stochastic gradient descent (SGD) algorithm for the deep ReLU nets training (called ReLU-SGD pair), the proposed sigmoid-ADMM pair is practically stable with respect to the algorithmic hyperparameters including the learning rate, initial schemes and the pro-processing of the input data. Moreover, we find that to approximate and learn simple but important functions the proposed sigmoid-ADMM pair numerically outperforms the ReLU-SGD pair.

No abstract uploaded!

We prove that uniform random quadrangulations of the sphere with$n$faces, endowed with the usual graph distance and renormalized by$n$^{−1/4}, converge as$n$→$∞$in distribution for the Gromov–Hausdorff topology to a limiting metric space. We validate a conjecture by Le Gall, by showing that the limit is (up to a scale constant) the so-called$Brownian map$, which was introduced by Marckert–Mokkadem and Le Gall as the most natural candidate for the scaling limit of many models of random plane maps. The proof relies strongly on the concept of$geodesic stars$in the map, which are configurations made of several geodesics that only share a common endpoint and do not meet elsewhere.

The p-th moment matrix is defined for a real random vector, generalizing the classical covariance matrix. Sharp inequalities relating the p-th moment and Renyi entropy are established, generalizing the classical inequality relating the second moment and the Shannon entropy. The extremal distributions for these inequalities are completely characterized.

We study the Newton polygon jumping locus of a Mumford family in char p. Our main result says that, under a mild assumption on p, the jumping locus consists of only supersingular points and its cardinality is equal to (p^r − 1)(g − 1), where r is the degree of the defining field of the base curve of a Mumford family in char p and g is the genus of the curve. The underlying technique is the p-adic Hodge theory.