No abstract uploaded!

The stochastic gradient (SG) method can quickly solve a problem with a large number of components in the objective, or a stochastic optimization problem, to a moderate accuracy. The block coordinate descent/update (BCD) method, on the other hand, can quickly solve problems with multiple (blocks of) variables. This paper introduces a method that combines the great features of SG and BCD for problems with many components in the objective and with multiple (blocks of) variables. This paper proposes a block SG (BSG) method for both convex and nonconvex programs. BSG generalizes SG by updating all the blocks of variables in the Gauss–Seidel type (updating the current block depends on the previously updated block), in either a fixed or randomly shuffled order. Although BSG has slightly more work at each iteration, it typically outperforms SG because of BSG’s Gauss–Seidel updates and larger step sizes, the latter of which are determined by the smaller per-block Lipschitz constants. The convergence of BSG is established for both convex and nonconvex cases. In the convex case, BSG has the same order of convergence rate as SG. In the nonconvex case, its convergence is established in terms of the expected violation of a first-order optimality condition. In both cases our analysis is nontrivial since the typical unbiasedness assumption no longer holds. BSG is numerically evaluated on the following problems: stochastic least squares and logistic regression, which are convex, and low-rank tensor recovery and bilinear logistic regression, which are nonconvex. On the convex problems, BSG performed significantly better than SG. On the nonconvex problems, BSG significantly outperformed the deterministic BCD method because the latter tends to stagnate early near local minimizers. Overall, BSG inherits the benefits of both SG approximation and block coordinate updates and is especially useful for solving large-scale nonconvex problems.

The immersed boundary method is one of the most useful computational methods in studying fluid structure interaction. On the other hand, the Immersed Boundary method is also known to require small time steps to maintain stability when solved with an explicit method. Many implicit or approximately implicit methods have been proposed in the literature to remove this severe time step stability constraint, but none of them give satisfactory performance. In this paper, we propose an efficient semi-implicit scheme to remove this stiffness from the immersed boundary method for the Navier–Stokes equations. The construction of our semi-implicit scheme consists of two steps. First, we obtain a semiimplicit discretization which is proved to be unconditionally stable. This unconditionally stable semi-implicit scheme is still quite expensive to implement in practice. Next, we apply the small scale decomposition to the unconditionally stable semi-implicit scheme to construct our efficient semi-implicit scheme. Unlike other implicit or semi-implicit schemes proposed in the literature, our semi-implicit scheme can be solved explicitly in the spectral space. Thus the computational cost of our semi-implicit schemes is comparable to that of an explicit scheme. Our extensive numerical experiments show that our semiimplicit scheme has much better stability property than an explicit scheme. This offers a substantial computational saving in using the immersed boundary method.

This article focuses on the discrete double-curl operator arising in the Maxwell equation that models three dimensional photonic crystals with face centered cubic lattice. The discrete double-curl operator is the degenerate coefficient matrix of the generalized eigenvalue problems (GEVP) due to the Maxwell equation. We derive an eigendecomposition of the degenerate coefficient matrix and explore an explicit form of orthogonal basis for the range and null spaces of this matrix. To solve the GEVP, we apply these theoretical results to project the GEVP to a standard eigenvalue problem (SEVP), which involves only the eigenspace associated with the nonzero eigenvalues of the GEVP and therefore the zero eigenvalues are excluded and will not degrade the computational efficiency. This projected SEVP can be solved efficiently by the inverse Lanczos method. The linear systems within the inverse Lanczos method are well-conditioned and can be solved efficiently by the conjugate gradient method without using a preconditioner. We also demonstrate how two forms of matrix-vector multiplications, which are the most costly part of the inverse Lanczos method, can be computed by fast Fourier transformation due to the eigendecomposition to significantly reduce the computation cost. Integrating all of these findings and techniques, we obtain a fast eigenvalue solver. The solver has been implemented by MATLAB and successfully solves each of a set of $5.184$ million dimension eigenvalue problems within $50$ to $104$ minutes on a workstation with two Intel Quad-C ore Xeon X5687 3.6 GHz CPUs.

No abstract uploaded!