Learning rate is arguably the most important hyper-parameter to tune when training a neural network. As manually setting right learning rate remains a cumbersome process, adaptive learning rate algorithms aim at automating such a process. Motivated by the success of the Barzilai–Borwein (BB) step-size method in many gradient descent methods for solving convex problems, this paper aims at investigating the potential of the BB method for training neural networks. With strong motivation from related convergence analysis, the BB method is generalized to adaptive learning rate of mini-batch gradient descent. The experiments showed that, in contrast to many existing methods, the proposed BB method is highly insensitive to initial learning rate, especially in terms of generalization performance. Also, the BB method showed its advantages on both learning speed and generalization performance over other available methods.

An elliptic equation arising from the study o fstatic solutions with prescribed zeros and poles of the Einstein equations coupled with the classical sigma model and an Abelian gauge field, is considered. We classify the solutions and establish the uniqueness of radially symmetric solutions. We also complete a classification of symmetric solutions of an elliptic equation on the sphere.

The continuity of weak solutions of elliptic partial differential equations $$div \mathcal{A}(x,\nabla u) = 0$$ is considered under minimal structure assumptions. The main result guarantees the continuity at the point$x$_{0}for weakly monotone weak solutions if the structure of$A$is controlled in a sequence of annuli $$B(x_0 ,R_j )\backslash \bar B(x_0 ,r_j )$$ with uniformly bounded ratio$R$_{$j$}$/r$_{$j$}such that lim_{$j→∞$}$R$_{$j$}=0. As a consequence, we obtain a sufficient condition for the continuity of mappings of finite distortion.

In this paper,we establish the convergence of the proximal alternating direction method of multipliers (ADMM) and block coordinate descent (BCD) method for nonseparable minimization models with quadratic coupling terms. The novel convergence results presented in this paper answer several open questions that have been the subject of considerable discussion. We firstly extend the 2-block proximal ADMM to linearly constrained convex optimization with a coupled quadratic objective function, an area where theoretical understanding is currently lacking, and prove that the sequence generated by the proximal ADMM converges in point-wise manner to a primal-dual solution pair. Moreover, we apply randomly permuted ADMM (RPADMM) to nonseparable multi-block convex optimization, and prove its expected convergence for a class of nonseparable quadratic programming problems. When the linear constraint vanishes, the 2-block proximal ADMM and RPADMM reduce to the 2-block cyclic proximal BCD method and randomly permuted BCD (RPBCD). Our study provides the first iterate convergence result for 2-block cyclic proximal BCD without assuming the boundedness of the iterates. We also theoretically establish the expected iterate convergence result concerning multi-block RPBCD for convex quadratic optimization. In addition, we demonstrate that RPBCD may have a worse convergence rate than cyclic proximalBCDfor 2-block convex quadratic minimization problems. Although the results on RPADMM and RPBCD are restricted to quadratic minimization models, they provide some interesting insights: (1) random permutation makes ADMM and BCD more robust for multi-block convex minimization problems; (2) cyclic BCD may outperform RPBCD for “nice” problems, and RPBCD should be applied with caution when solving general convex optimization problems especially with a few blocks.

We study the positive representations Pλ of split real quantum groups Uqq (gℝ) restricted to the Borel subalgebra Uqq (bℝ). We prove that the restriction is independent of the parameter λ. Furthermore, we prove that it can be constructed from the GNS-representation of the multiplier Hopf algebra UqqC ∗ (b ℝ) defined earlier, which allows us to decompose their tensor product using the theory of the “multiplicative unitary”. In particular, the quantum mutation operator can be constructed from the multiplicity module, which will be an essential ingredient in the construction of quantum higher Teichmüller theory from the perspective of representation theory, generalizing earlier work by Frenkel-Kim.