The stochastic gradient (SG) method can quickly solve a problem with a large number of components in the objective, or a stochastic optimization problem, to a moderate accuracy. The block coordinate descent/update (BCD) method, on the other hand, can quickly solve problems with multiple (blocks of) variables. This paper introduces a method that combines the great features of SG and BCD for problems with many components in the objective and with multiple (blocks of) variables. This paper proposes a block SG (BSG) method for both convex and nonconvex programs. BSG generalizes SG by updating all the blocks of variables in the Gauss–Seidel type (updating the current block depends on the previously updated block), in either a fixed or randomly shuffled order. Although BSG has slightly more work at each iteration, it typically outperforms SG because of BSG’s Gauss–Seidel updates and larger step sizes, the latter of which are determined by the smaller per-block Lipschitz constants. The convergence of BSG is established for both convex and nonconvex cases. In the convex case, BSG has the same order of convergence rate as SG. In the nonconvex case, its convergence is established in terms of the expected violation of a first-order optimality condition. In both cases our analysis is nontrivial since the typical unbiasedness assumption no longer holds. BSG is numerically evaluated on the following problems: stochastic least squares and logistic regression, which are convex, and low-rank tensor recovery and bilinear logistic regression, which are nonconvex. On the convex problems, BSG performed significantly better than SG. On the nonconvex problems, BSG significantly outperformed the deterministic BCD method because the latter tends to stagnate early near local minimizers. Overall, BSG inherits the benefits of both SG approximation and block coordinate updates and is especially useful for solving large-scale nonconvex problems.

No abstract uploaded!

We consider Hermitian and symmetric random band matrices $H = (h_{xy})$ in $d \geq 1$ dimensions. The matrix entries $h_{xy}$, indexed by $x,y \in (\bZ/L\bZ)^d$, are independent, centred random variables with variances $s_{xy} = \E |h_{xy}|^2$. We assume that $s_{xy}$ is negligible if $|x-y|$ exceeds the band width $W$. In one dimension we prove that the eigenvectors of $H$ are delocalized if $W\gg L^{4/5}$. We also show that the magnitude of the matrix entries $\abs{G_{xy}}^2$ of the resolvent $G=G(z)=(H-z)^{-1}$ is self-averaging and we compute $\E \abs{G_{xy}}^2$. We show that, as $L\to\infty$ and $W\gg L^{4/5}$, the behaviour of $\E |G_{xy}|^2$ is governed by a diffusion operator whose diffusion constant we compute. Similar results are obtained in higher dimensions.

No abstract uploaded!

Surface parameterizations have been widely applied to digital geometry processing. In this paper, we propose an efficient conformal energy minimization (CEM) algorithm for computing conformal parameterizations of simply-connected open surfaces with a very small angular distortion and a highly improved computational efficiency. In addition, we generalize the proposed CEM algorithm to computing conformal parameterizations of multiply-connected surfaces. Furthermore, we prove the existence of a nontrivial accumulation point of the proposed CEM algorithm under some mild conditions. Several numerical results show the efficiency and robustness of the CEM algorithm comparing to the existing state-of-the-art algorithms. An application of the CEM on the surface morphing between simply-connected open surfaces is demonstrated thereafter. Thanks to the CEM algorithm, the whole computations for the surface morphing can be performed efficiently and robustly.