In this paper, we propose a new stabilized linear finite element method for solving reaction-convection-diffusion equations with arbitrary magnitudes of reaction and diffusion. The key feature of the new method is that the test function in the stabilization term is taken in the adjoint-operator-like form $-\varepsilon \Delta v- ({\mbf a}\cdot\nabla v)/\gamma+\sigma v$, where the parameter $\gamma$ is appropriately designed to adjust the convection strength to achieve high accuracy and stability. We derive the stability estimates for the finite element solutions and establish the explicit dependence of $L^2$ and $H^1$ error bounds on the diffusivity, modulus of the convection field, reaction coefficient and the mesh size. The analysis shows that the proposed method is suitable for a wide range of mesh P\'{e}clet numbers and mesh Damk\"{o}hler numbers. More specifically, if the diffusivity $\varepsilon$ is sufficiently small with $\varepsilon < \Vert{\mbf a}\Vert h$ and the reaction coefficient $\sigma$ is large enough such that $\Vert{\mbf a}\Vert < \sigma h$, then the method exhibits optimal convergence rates in both $L^2$ and $H^1$ norms. However, for a small reaction coefficient satisfying $\Vert{\mbf a}\Vert \ge \sigma h$, the method behaves like the well-known streamline upwind/Petrov-Galerkin formulation of Brooks and Hughes. Several numerical examples exhibiting boundary or interior layers are given to demonstrate the high performance of the proposed method. Moreover, we apply the developed method to time-dependent reaction-convection-diffusion problems and simulation results show the efficiency of the approach.

This paper focuses on coordinate update methods, which are useful for solving problems involving large or high-dimensional datasets. They decompose a problem into simple subproblems, where each updates one, or a small block of, variables while fixing others. These methods can deal with linear and nonlinear mappings, smooth and nonsmooth functions, as well as convex and nonconvex problems. In addition, they are easy to parallelize. The great performance of coordinate update methods depends on solving simple sub-problems. To derive simple subproblems for several new classes of applications, this paper systematically studies coordinate-friendly operators that perform low-cost coordinate updates. Based on the discovered coordinate friendly operators, as well as operator splitting techniques, we obtain new coordinate update algorithms for a variety of problems in machine learning, image processing, as well as sub-areas of optimization. Several problems are treated with coordinate update for the first time in history. The obtained algorithms are scalable to large instances through parallel and even asynchronous computing. We present numerical examples to illustrate how effective these algorithms are.

In this paper, we propose a novel and simple technique to construct effective difference schemes for solving systems of singularly perturbed convection-diffusionreaction equations, whose solutions may display boundary or interior layers. We illustrate the technique by taking the Il’in-Allen-Southwell scheme for 1-D scalar equations as a basis to derive a formally second-order scheme for 1-D coupled systems and then extend the scheme to 2-D case by employing an alternating direction approach. Numerical examples are given to demonstrate the high performance of the obtained scheme on uniform meshes as well as piecewise-uniform Shishkin meshes.

This article studies the problem of image restoration of observed images corrupted by impulse noise and mixed Gaussian impulse noise. Since the pixels damaged by impulse noise contain no information about the true image, how to find this set correctly is a very important problem. We propose two methods based on blind inpainting and $\ell_0$ minimization that can simultaneously find the damaged pixels and restore the image. By iteratively restoring the image and updating the set of damaged pixels, these methods have better performance than other methods, as shown in the experiments. In addition, we provide convergence analysis for these methods; these algorithms will converge to coordinatewise minimum points. In addition, they will converge to local minimum points (or with probability one) with some modifications in the algorithms.

Finding a fixed point to a nonexpansive operator, i.e., x = Tx, abstracts many problems in numerical linear algebra, optimization, and other areas of data sciences. To solve xed- point problems, we propose ARock, an algorithmic framework in which multiple agents (machines, processors, or cores) update x in an asynchronous parallel fashion. Asynchrony is crucial to parallel computing since it reduces synchronization wait, relaxes communication bottleneck, and thus speeds up computing significantly. At each step of ARock, an agent updates a randomly selected coordinate xi based on possibly out-of-date information on x. The agents share x through either global memory or communication. If writing xi is atomic, the agents can read and write x without memory locks. We prove that if the nonexpansive operator T has a fixed point, then with probability one, ARock generates a sequence that converges to a fixed point of T. Our conditions on T and step sizes are weaker than comparable work. Linear convergence is obtained under suitable assumptions. We propose special cases of ARock for linear systems, convex optimization, machine learning, as well as distributed and decentralized consensus problems. Numerical experiments of solving sparse logistic regression problems are presented.

Motivated by big data applications, first-order methods have been extremely popular in recent years. However, naive gradient methods generally converge slowly. Hence, much efforts have been made to accelerate various first-order methods. This paper proposes two accelerated methods towards solving structured linearly constrained convex programming, for which we assume composite convex objective that is the sum of a differentiable function and a possibly nondifferentiable one. The first method is the accelerated linearized augmented Lagrangian method (LALM). At each update to the primal variable, it allows linearization to the differentiable function and also the augmented term, and thus it enables easy subproblems. Assuming merely convexity, we show that LALM owns O(1/t) convergence if parameters are kept fixed during all the iterations and can be accelerated to O(1/t^2) if the parameters are adapted, where t is the number of total iterations. The second method is the accelerated linearized alternating direction method of multipliers (LADMM). In addition to the composite convexity, it further assumes two-block structure on the objective. Different from classic ADMM, our method allows linearization to the objective and also augmented term to make the update simple. Assuming strong convexity on one block variable, we show that LADMM also enjoys O(1=t2) convergence with adaptive parameters. This result is a significant improvement over that in [Goldstein et. al, SIIMS'14], which requires strong convexity on both block variables and no linearization to the objective or augmented term. Numerical experiments are performed on quadratic programming, image denoising, and support vector machine. The proposed accelerated methods are compared to nonaccelerated ones and also existing accelerated methods. The results demonstrate the validness of acceleration and superior performance of the proposed methods over existing ones.

Nonconvex optimization arises in many areas of computational science and engineering. However, most nonconvex optimization algorithms are only known to have local convergence or subsequence convergence properties. In this paper, we propose an algorithm for nonconvex optimization and establish its global convergence (of the whole sequence) to a critical point. In addition, we give its asymptotic convergence rate and numerically demonstrate its efficiency. In our algorithm, the variables of the underlying problem are either treated as one block or multiple disjoint blocks. It is assumed that each non-differentiable component of the objective function, or each constraint, applies only to one block of variables. The differentiable components of the objective function, however, can involve multiple blocks of variables together. Our algorithm updates one block of variables at a time by minimizing a certain prox-linear surrogate, along with an extrapolation to accelerate its convergence. The order of update can be either deterministically cyclic or randomly shuffled for each cycle. In fact, our convergence analysis only needs that each block be updated at least once in every fixed number of iterations. We show its global convergence (of the whole sequence) to a critical point under fairly loose conditions including, in particular, the Kurdyka–Łojasiewicz condition, which is satisfied by a broad class of nonconvex/nonsmooth applications. These results, of course, remain valid when the underlying problem is convex.We apply our convergence results to the coordinate descent iteration for non-convex regularized linear regression, as well as amodified rank-one residue iteration for nonnegative matrix factorization. We show that both applications have global convergence. Numerically,we tested our algorithm on nonnegative matrix and tensor factorization problems, where random shuffling clearly improves the chance to avoid low-quality local solutions.

This paper considers regularized block multiconvex optimization, where the feasible set and objective function are generally nonconvex but convex in each block of variables. It also accepts nonconvex blocks and requires these blocks to be updated by proximal minimization. We review some interesting applications and propose a generalized block coordinate descent method. Under certain conditions, we show that any limit point satisfies the Nash equilibrium conditions. Furthermore, we establish global convergence and estimate the asymptotic convergence rate of the method by assuming a property based on the Kurdyka-Lojasiewicz inequality. The proposed algorithms are tested on nonnegative matrix and tensor factorization, as well as matrix and tensor recovery from incomplete observations. The tests include synthetic data and hyperspectral data, as well as image sets from the CBCL and ORL databases. Compared to the existing state-of-the-art algorithms, the proposed algorithms demonstrate superior performance in both speed and solution quality. The MATLAB code of nonnegative matrix/tensor decomposition and completion, along with a few demos, are accessible from the authors' homepages.

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!

Peskin's Immersed Boundary (IB) method is one of the most popular numerical methods for many years and has been applied to problems in mathematical biology, fluid mechanics, material sciences, and many other areas. Peskiness IB method is associated with discrete delta functions. It is believed that the IB method is first order accurate in the $L^{\infty}$ norm. But almost no rigorous proof could be found in the literature until recently [Mori, Comm. Pure. Appl. Math: 61:2008] in which the author showed that the velocity is indeed first order accurate for the Stokes equations with a periodic boundary condition. In this paper, we show first order convergence with a $\log h$ factor of the IB method for elliptic interface problems with Dirichlet boundary conditions.

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!

No abstract uploaded!