In this paper,we establish the convergence of the proximal alternating direction method of multipliers (ADMM) and block coordinate descent (BCD) method for nonseparable minimization models with quadratic coupling terms. The novel convergence results presented in this paper answer several open questions that have been the subject of considerable discussion. We firstly extend the 2-block proximal ADMM to linearly constrained convex optimization with a coupled quadratic objective function, an area where theoretical understanding is currently lacking, and prove that the sequence generated by the proximal ADMM converges in point-wise manner to a primal-dual solution pair. Moreover, we apply randomly permuted ADMM (RPADMM) to nonseparable multi-block convex optimization, and prove its expected convergence for a class of nonseparable quadratic programming problems. When the linear constraint vanishes, the 2-block proximal ADMM and RPADMM reduce to the 2-block cyclic proximal BCD method and randomly permuted BCD (RPBCD). Our study provides the first iterate convergence result for 2-block cyclic proximal BCD without assuming the boundedness of the iterates. We also theoretically establish the expected iterate convergence result concerning multi-block RPBCD for convex quadratic optimization. In addition, we demonstrate that RPBCD may have a worse convergence rate than cyclic proximalBCDfor 2-block convex quadratic minimization problems. Although the results on RPADMM and RPBCD are restricted to quadratic minimization models, they provide some interesting insights: (1) random permutation makes ADMM and BCD more robust for multi-block convex minimization problems; (2) cyclic BCD may outperform RPBCD for “nice” problems, and RPBCD should be applied with caution when solving general convex optimization problems especially with a few blocks.

Many optimization algorithms converge to stationary points. When the underlying problem is nonconvex, they may get trapped at local minimizers and occasionally stagnate near saddle points. We propose the Run-and-Inspect Method, which adds an “inspect” phase to existing algorithms that helps escape from non-global stationary points. The inspection samples a set of points in a radius R around the current point. When a sample point yields a sufficient decrease in the objective, we resume an existing algorithm from that point. If no sufficient decrease is found, the current point is called an approximate R-local minimizer. We show that an R-local minimizer is globally optimal, up to a specific error depending on R, if the objective function can be implicitly decomposed into a smooth convex function plus a restricted function that is possibly nonconvex, nonsmooth. Therefore, for such nonconvex objective functions, verifying global optimality is fundamentally easier. For high-dimensional problems, we introduce blockwise inspections to overcome the curse of dimensionality while still maintaining optimality bounds up to a factor equal to the number of blocks. Our method performs well on a set of artificial and realistic nonconvex problems by coupling with gradient descent, coordinate descent, EM, and prox-linear algorithms.

This paper discusses how to solve filtering problem for a class of continuous nonlinear time-varying systems via Duncan-Mortensen-Zakai (DMZ) equation. In this paper, the original DMZ equation is changed into Kolmogorov forward equation (KFE) by exponential transformations in each time interval, and then under some assumptions, the KFE can be transformed into time-varying Schr\"odinger equation which can be solved explicitly. The novelty of this paper lies in how to transform the KFE into Schr\"odinger equation. As a direct application, the results of \cite{yau 2004} are extended for time-varying Yau systems.

The free boundary value problem in obstacle problem for von Krmn equations is studied. By using the method of complementarity analysis, Rockafellar's theory of duality is generalized to the nonlinear variational problems and a complementarity theory of obstacle problem for von Krmn plates is established. We prove that the uniqueness and existence of solution directly depend on a complementary gap function. Moreover, a generalized dual extreme principle is established. We prove that the nonlinear primal variational inequality problem is eventually equivalent to a semi-quadratic dual optimization problem defined on a statically admissible space. This equivalence can be used to develop an effective numerical method for solving nonlinear free boundary value problems.

In this paper, we study the rate of convergence of the cyclic projection algorithm applied to finitely many basic semialgebraic convex sets. We establish an explicit convergence rate estimate which relies on the maximum degree of the polynomials that generate the basic semialgebraic convex sets and the dimension of the underlying space. We achieve our results by exploiting the algebraic structure of the basic semialgebraic convex sets.