Anderson mixing (AM) is a powerful acceleration method for fixed-point iterations, but its computation requires storing many historical iterations. The extra memory footprint can be prohibitive when solving high-dimensional problems in a resource-limited machine. To reduce the memory overhead, we propose a novel class of short-term recurrence AM methods (ST-AM). The ST-AM methods only store two previous iterations with cheap corrections. We prove that the basic version of ST-AM is equivalent to the full-memory AM in strongly convex quadratic optimization, and with minor changes it has local linear convergence for solving general nonlinear fixed-point problems. We further analyze the convergence properties of the regularized ST-AM for nonconvex (stochastic) optimization. Finally, we apply ST-AM to several applications including solving root-finding problems and training neural networks. Experimental results show that ST-AM is competitive with the long-memory AM and outperforms many existing optimizers.

No abstract uploaded!

Representations of the sl (n+ 1, C) Lie algebras are constructed with the help of canonical (boson) realisations of these algebras. For every weight Lambda on the standard Cartan subalgebra of sl (n+ 1, C) the authors obtain a representation rho Lambda (n+ 1)(called the maximal representation) which contains an irreducible subrepresentation with Lambda as the highest weight. It is shown that for a major part of the weights Lambda the representations rho Lambda (n+ 1) themselves are irreducible. The standard construction of the highest-weight representations of semi-simple Lie algebras is based on the so-called elementary representations; comparing with them, the authors maximal representations are given in the explicit form.

No abstract uploaded!

No abstract uploaded!