We extend the RayleighRitz method to the eigen-problem of periodic matrix pairs. Assuming that the deviations of the desired periodic eigenvectors from the corresponding periodic subspaces tend to zero, we show that there exist periodic Ritz values that converge to the desired periodic eigenvalues unconditionally, yet the periodic Ritz vectors may fail to converge. To overcome this potential problem, we minimize residuals formed with periodic Ritz values to produce the refined periodic Ritz vectors, which converge under the same assumption. These results generalize the corresponding well-known ones for RayleighRitz approximations and their refinement for non-periodic eigen-problems. In addition, we consider a periodic Arnoldi process which is particularly efficient when coupled with the RayleighRitz method with refinement. The numerical results illustrate that the refinement procedure produces excellent

We propose an algorithm for the state feedback pole assignment problem. The algorithm is the first of its kind, making use of the Schur form and minimizing the departure from normality of the closed-loop poles by Newtons method. Eleven illustrative examples, comparing our algorithm with three other existing ones, are given.

In Chu (Syst Control Lett 56:303314, 2007), the pole assignment problem was considered for the control system \dot{x} = Ax + Bu with linear state-feedback \dot{x} = Ax + Bu An algorithm using the Schur form has been proposed, producing good suboptimal solutions which can be refined further using optimization. In this paper, the algorithm is improved, incorporating the minimization of the feedback gain \dot{x} = Ax + Bu It is also extended for the pole assignment of the descriptor system \dot{x} = Ax + Bu with linear state- and derivative-feedback \dot{x} = Ax + Bu Newton refinement for the solutions is discussed and several illustrative numerical examples are presented.

We consider the solution of large-scale discrete-time algebraic Riccati equations with numerically low-ranked solutions. The structure-preserving doubling algorithm will be adapted, with the iterates for A not explicitly computed but in the recursive form Ak= A2 k 1 D (1) k

In this paper, we present an efficient QR algorithm for solving the linear response eigenvalue problem H x= x, where H is -symmetric with respect to 0= diag (I n, I n). Based on newly introduced -orthogonal transformations, the QR algorithm preserves the -symmetric structure of H throughout the whole process, and thus guarantees the computed eigenvalues to appear pairwise (, ) as they should. With the help of a newly established implicit -orthogonality theorem, we incorporate the implicit multi-shift technique to accelerate the convergence of the QR algorithm. Numerical experiments are given to show the effectiveness of the algorithm.

In this paper, we propose the palindromic doubling algorithm (PDA) for the palindromic generalized eigenvalue problem (PGEP) A x= Ax. We establish a complete convergence theory of the PDA for PGEPs without unimodular eigenvalues, or with unimodular eigenvalues of partial multiplicities two (one or two for eigenvalue 1). Some important applications from the vibration analysis and the optimal control for singular descriptor linear systems will be presented to illustrate the feasibility and efficiency of the PDA.

We discuss a variant of the multi-task <i>n</i>-vehicle exploration problem. Instead of requiring an optimal permutation of vehicles in every group, the new problem requires all vehicles in a group to arrive at the same destination. Given <i>n</i> tasks with assigned consume-time and profit, it may also be viewed as a maximization of every processors average profit. Further, we propose a new kind of partition problem in fractional form and analyze its computational complexity. By regarding fractional partition as a special case, we prove that the average profit maximization problem is NP-hard when the number of processors is fixed and it is strongly NPhard in general. At last, a pseudo-polynomial time algorithm for the average profit maximization problem and the fractional partition problem is presented, using the idea of the pseudo-polynomial time algorithm for the classical partition problem.

High-dimensional data contain not only redundancy but also noises produced by the sensors. These noises are usually non-Gaussian distributed. The metrics based on Euclidean distance are not suitable for these situations in general. In order to select the useful features and combat the adverse effects of the noises simultaneously, a robust sparse subspace learning method in unsupervised scenario is proposed in this paper based on the maximum correntropy criterion that shows strong robustness against outliers. Furthermore, an iterative strategy based on half quadratic and an accelerated block coordinate update is proposed. The convergence analysis of the proposed method is also carried out to ensure the convergence to a reliable solution. Extensive experiments are conducted on real-world data sets to show that the new method can filter out the outliers and outperform several state-of-the-art unsupervised

The support vector machine (SVM) has been used in a wide variety of classification problems. The original SVM uses the hinge loss function, which is non-differentiable and makes the problem difficult to solve in particular for regularized SVMs, such as with 1 -regularization. This paper considers the Huberized SVM (HSVM), which uses a differentiable approximation of the hinge loss function. We first explore the use of the proximal gradient (PG) method to solving binary-class HSVM (B-HSVM) and then generalize it to multi-class HSVM (M-HSVM). Under strong convexity assumptions, we show that our algorithm converges linearly. In addition, we give a finite convergence result about the support of the solution, based on which we further accelerate the algorithm by a two-stage method. We present extensive numerical experiments on both synthetic and real datasets which demonstrate the superiority of our

Stochastic gradient (SG) method has been popularly applied to solve optimization problems with objective that is stochastic or an average of many functions. Most existing works on SG assume that the underlying problem is unconstrained or has an easy-to-project constraint set. In this paper, we consider problems that have a stochastic objective and also many functional constraints. For such problems, it could be extremely expensive to project a point to the feasible set, or even compute subgradient and/or function value of all constraint functions. To find solutions of these problems, we propose a novel SG method based on the augmented Lagrangian function. Within every iteration, it inquires a stochastic subgradient of the objective, a subgradient and function value of one randomly sampled constraint function, and function value of another sampled constraint function. Hence, the per-iteration complexity is low. We establish its convergence rate for convex and also strongly convex problems. It can achieve the optimal O (1/\sqrt {k}) convergence rate for convex case and nearly optimal O (1/\sqrt {k}) rate for strongly convex case. Numerical experiments on quadratically constrained quadratic programming are conducted to demonstrate its efficiency.

Recent several years have witnessed the surge of asynchronous (async-) parallel computing methods due to the extremely big data involved in many modern applications and also the advancement of multi-core machines and computer clusters. In optimization, most works about async-parallel methods are on unconstrained problems or those with block separable constraints. In this paper, we propose an async-parallel method based on block coordinate update (BCU) for solving convex problems with <i>nonseparable</i> linear constraint. Running on a single node, the method becomes a novel randomized primaldual BCU for multi-block affinely constrained problems. For these problems, GaussSeidel cyclic primaldual BCU is not guaranteed to converge to an optimal solution if no additional assumptions, such as strong convexity, are made. On the contrary, assuming convexity and existence of a primaldual

Recent years have witnessed the rapid development of block coordinate update (BCU) methods, which are particularly suitable for problems involving large-sized data and/or variables. In optimization, BCU first appears as the coordinate descent method that works well for smooth problems or those with separable nonsmooth terms and/or separable constraints. As nonseparable constraints exist, BCU can be applied under primal-dual settings. In the literature, it has been shown that for weakly convex problems with nonseparable linear constraints, BCU with fully Gauss--Seidel updating rule may fail to converge and that with fully Jacobian rule can converge sublinearly. However, empirically the method with Jacobian update is usually slower than that with Gauss--Seidel rule. To maintain their advantages, we propose a hybrid Jacobian and Gauss--Seidel BCU method for solving linearly constrained multiblock

Block coordinate update (BCU) methods enjoy low per-update computational complexity because every time only one or a few block variables would need to be updated among possibly a large number of blocks. They are also easily parallelized and thus have been particularly popular for solving problems involving large-scale dataset and/or variables. In this paper, we propose a primaldual BCU method for solving linearly constrained convex program with multi-block variables. The method is an accelerated version of a primaldual algorithm proposed by the authors, which applies randomization in selecting block variables to update and establishes an <i>O</i>(1/<i>t</i>) convergence rate under convexity assumption. We show that the rate can be accelerated to O ( 1 / t 2 ) if the objective is strongly convex. In addition, if one block variable is independent of the others in the objective, we then show that the algorithm can

In this paper, we propose a randomized primaldual proximal block coordinate updating framework for a general multi-block convex optimization model with coupled objective function and linear constraints. Assuming mere convexity, we establish its <i>O</i>(1/<i>t</i>) convergence rate in terms of the objective value and feasibility measure. The framework includes several existing algorithms as special cases such as a primaldual method for bilinear saddle-point problems (PD-S), the proximal Jacobian alternating direction method of multipliers (Prox-JADMM) and a randomized variant of the ADMM for multi-block convex optimization. Our analysis recovers and/or strengthens the convergence properties of several existing algorithms. For example, for PD-S our result leads to the same order of convergence rate without the previously assumed boundedness condition on the constraint sets, and for Prox-JADMM the new

This paper introduces algorithms for the decentralized low-rank matrix completion problem. Assume a low-rank matrix W = [W ₁ ,W ₂ , ...,W _L ]. In a network, each agent observes some entries of W  . In order to recover the unobserved entries of W via decentralized computation, we factorize the unknown matrix W as the product of a public matrix X, common to all agents, and a private matrix Y = [Y ₁ ,Y ₂ , ...,Y _L ], where Y  is held by agent . Each agent alternatively updates Y  and its local estimate of X while communicating with its neighbors toward a consensus on the estimate. Once this consensus is (nearly) reached throughout the network, each agent recovers W  = XY  , and thus W is recovered. The communication cost is scalable to the number of agents, and W  and Y  are kept private to agent to a certain extent. The algorithm is accelerated by extrapolation and compares favorably to the centralized

This paper focuses on coordinate update methods, which are useful for solving problems involving large or high-dimensional datasets. They decompose a problem into simple subproblems, where each updates one, or a small block of, variables while fixing others. These methods can deal with linear and nonlinear mappings, smooth and nonsmooth functions, as well as convex and nonconvex problems. In addition, they are easy to parallelize.

Higher-order low-rank tensors naturally arise in many applications including hyperspectral data recovery, video inpainting, seismic data recon-struction, and so on. We propose a new model to recover a low-rank tensor by simultaneously performing low-rank matrix factorizations to the all-mode ma-tricizations of the underlying tensor. An alternating minimization algorithm is applied to solve the model, along with two adaptive rank-adjusting strategies when the exact rank is not known.

This paper introduces an algorithm for the nonnegative matrix factorization-and-completion problem, which aims to find nonnegative low-rank matrices <i>X</i> and <i>Y</i> so that the product <i>XY</i> approximates a nonnegative data matrix <i>M</i> whose elements are partially known (to a certain accuracy). This problem aggregates two existing problems: (i) nonnegative matrix factorization where all entries of <i>M</i> are given, and (ii) low-rank matrix completion where nonnegativity is not required. By taking the advantages of both nonnegativity and low-rankness, one can generally obtain superior results than those of just using one of the two properties. We propose to solve the non-convex constrained least-squares problem using an algorithm based on the classical alternating direction augmented Lagrangian method. Preliminary convergence properties of the algorithm and numerical simulation results are presented. Compared to

In this paper, we first study \ell_q minimization and its associated iterative reweighted algorithm for recovering sparse vectors. Unlike most existing work, we focus on <i>unconstrained</i> \ell_q minimization, for which we show a few advantages on noisy measurements and/or approximately sparse vectors. Inspired by the results in [Daubechies et al., <i>Comm. Pure Appl. Math.</i>, 63 (2010), pp. 1--38] for <i>constrained</i> \ell_q minimization, we start with a preliminary yet novel analysis for <i>unconstrained</i> \ell_q minimization, which includes convergence, error bound, and local convergence behavior. Then, the algorithm and analysis are extended to the recovery of low-rank matrices. The algorithms for both vector and matrix recovery have been compared to some state-of-the-art algorithms and show superior performance on recovering sparse vectors and low-rank matrices.

No abstract uploaded!

This work is devoted to the numerical computation of complex band structure $\k=\k(\omega)\in\mathbb C^3$ for positive $\omega$ of three dimensional isotropic dispersive or non-dispersive photonic crystals from the perspective of structured quadratic eigenvalue problems (QEPs). Our basic strategy is to fix two degrees of freedom in $\k\in\mathbb C^3$ and to view the remaining one as the eigenvalue of a quadratic operator pencil derived from Maxwell's equations. Then Yee's scheme is employed to discretize $\nabla\times$ and $\k\times$ operators in this quadratic operator pencil. Distinct from the others' works which either ignore or directly exploit the Hamiltonian structure of the spectrum of the resulting QEP, we reformulate this QEP into an equivalent $\top$-palindromic QEP to facilitate the use of superior structure-preserving algorithms. Ultimately we rely on the structured Arnoldi algorithm, namely the G$\top$SHIRA algorithm, to compute eigenvalues of a $\top$-skew-Hamiltonian pair which are near or in $[-2,2]$, a much narrower region than the whole positive real axis in the origin problem. Moreover, to accelerate the inner iterations of the G$\top$SHIRA algorithm, we propose the preconditioning technique, making most of the eigenmatrix, which can essentially be seen as the Kronecker product of three discrete Fourier transformation matrices, of the commutative discretized $\partial_x,\partial_y,\partial_z$ operators. The advantage of our method is discussed in detail and corroborated by several numerical results.

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!