The moment-entropy inequality shows that a contin- uous random variable with given second moment and maximal Shannon entropy must be Gaussian. Stam’s inequality shows that a continuous random variable with given Fisher information and minimal Shannon entropy must also be Gaussian. The Crame ́r- Rao inequality is a direct consequence of these two inequalities. In this paper the inequalities above are extended to Renyi entropy, p-th moment, and generalized Fisher information. Gen- eralized Gaussian random densities are introduced and shown to be the extremal densities for the new inequalities. An extension of the Crame ́r–Rao inequality is derived as a consequence of these moment and Fisher information inequalities.

entangled states. We construct sets of fewer than d orthogonal maximally entangled states which are not distinguished by one-way local operations and classical communication (LOCC) in the Hilbert space of d ⊗ d. The proof, based on the Fourier transform of an additive group, is very simple but quite effective. Simultaneously, our results give a general unified upper bound for the minimum number of one-way LOCC indistinguishable maximally entangled states. This improves previous results which only showed sets of N  d − 2 such states. Finally, our results also show that previous conjectures in Zhang et al. [Z.-C. Zhang, Q.-Y. Wen, F. Gao, G.-J. Tian, and T.-Q. Cao, Quant. Info. Proc. 13, 795 (2014)] are indeed correct.

The p-th moment matrix is defined for a real random vector, generalizing the classical covariance matrix. Sharp inequalities relating the p-th moment and Renyi entropy are established, generalizing the classical inequality relating the second moment and the Shannon entropy. The extremal distributions for these inequalities are completely characterized.

We study the minimization problem of a non-convex sparsity promoting penalty func- tion, the transformed l1 (TL1), and its application in compressed sensing (CS). The TL1 penalty inter- polates l0 and l1 norms through a nonnegative parameter a ∈ (0, +∞), similar to lp with p ∈ (0, 1], and is known to satisfy unbiasedness, sparsity and Lipschitz continuity properties. We first consider the constrained minimization problem, and discuss the exact recovery of l0 norm minimal solution based on the null space property (NSP). We then prove the stable recovery of l0 norm minimal solution if the sensing matrix A satisfies a restricted isometry property (RIP). We formulated a normalized problem to overcome the lack of scaling property of the TL1 penalty function. For a general sensing matrix A, we show that the support set of a local minimizer corresponds to linearly independent columns of A. Next, we present difference of convex algorithms for TL1 (DCATL1) in computing TL1-regularized con- strained and unconstrained problems in CS. The DCATL1 algorithm involves outer and inner loops of iterations, one time matrix inversion, repeated shrinkage operations and matrix-vector multiplications. The inner loop concerns an l1 minimization problem on which we employ the Alternating Direction Method of Multipliers (ADMM). For the unconstrained problem, we prove convergence of DCATL1 to a stationary point satisfying the first order optimality condition. In numerical experiments, we identify the optimal value a = 1, and compare DCATL1 with other CS algorithms on two classes of sensing ma- trices: Gaussian random matrices and over-sampled discrete cosine transform matrices (DCT). Among existing algorithms, the iterated reweighted least squares method based on l1/2 norm is the best in sparse recovery for Gaussian matrices, and the DCA algorithm based on l1 minus l2 penalty is the best for over-sampled DCT matrices. We find that for both classes of sensing matrices, the performance of DCATL1 algorithm (initiated with l1 minimization) always ranks near the top (if not the top), and is the most robust choice insensitive to the conditioning of the sensing matrix A. DCATL1 is also com- petitive in comparison with DCA on other non-convex penalty functions commonly used in statistics with two hyperparameters.

In this paper, we consider the harmonic extension problem, which is widely used in many applications of machine learning. We formulate the harmonic extension as solving a Laplace-Beltrami equation with Dirichlet boundary condition. We use the point integral method (PIM) to solve the Laplace-Beltrami equation. The basic idea of the PIM method is to approximate the Laplace equation using an integral equation, which is easy to be discretized from points. Based on the integral equation, we found that traditional graph Laplacian method (GLM) may fail to approximate the harmonic functions in the classical sense. For the Laplace-Beltrami equation with Dirichlet boundary, we can prove the convergence of the point integral method. The point integral method is also very easy to implement, which only requires a minor modification of the graph Laplacian. One important application of the harmonic extension in machine learning is semi-supervised learning. We run a popular semi-supervised learning algorithm by Zhu et al. over a couple of well-known datasets and compare the performance of the aforementioned approaches. Our experiments show the PIM performs the best. We also apply PIM to an image recovery problem and show it outperforms GLM. Finally, on a model problem of Laplace-Beltrami equation with Dirichlet boundary, we prove the convergence of the point integral method.

We formulate the Riemannian calculus of the probability set embedded with L^2-Wasserstein metric. This is an initial work of transport information geometry. Our investigation starts with the probability simplex (probability manifold) supported on vertices of a finite graph. The main idea is to embed the probability manifold as a submanifold of the positive measure space with a nonlinear metric tensor. Here the nonlinearity comes from the linear weighted Laplacian operator. By this viewpoint, we establish torsion-free Christoffel symbols, Levi-Civita connections, curvature tensors, and volume forms in the probability manifold by Euclidean coordinates. As a consequence, the Jacobi equation, Laplace-Beltrami and Hessian operators on the probability manifold are derived. These geometric computations are also provided in the infinite-dimensional density space (density manifold) supported on a finite-dimensional manifold. In particular, an identity is given connecting the Baker-{É}mery Γ2 operator (carr{é} du champ it{é}r{é}) by connecting Fisher-Rao information metric and optimal transport metric. Several examples are demonstrated.

We explain how the classical notions of Fisher information of a random variable and Fisher information matrix of a random vector can be extended to a much broader setting. We also show that Stam’s inequality for Fisher information and Shannon entropy, as well as the more generalized versions proved earlier by the authors, are all special cases of more general sharp inequalities satisfied by random vectors. The extremal random vectors, which we call generalized Gaussians, contain Gaussians as a limiting case but are noteworthy because they are heavy- tailed.

A unified approach is presented for establishing a broad class of Cram\'er-Rao inequalities for the location parameter, including, as special cases, the original inequality of Cram\'er and Rao, as well as an $L^p$ version recently established by the authors. The new approach allows for generalized moments and Fisher information measures to be defined by convex functions that are not necessarily homogeneous. In particular, it is shown that associated with any log-concave random variable whose density satisfies certain boundary conditions is a Cram\'er-Rao inequality for which the given log-concave random variable is the extremal. Applications to specific instances are also provided.

Poisson equation on point cloud with Dirichlet boundary condition plays important role in many problems. In this paper, we use the volume constraint proposed by Du et.al to handle the Dirichlet boundary condition in the point integral method for Poisson equation on point cloud. We prove that the solution given by volume constraint converges to the true solution as the point cloud converges to the underlying smooth manifold.

For a random quantum state on $\cH=\C^d \otimes \C^d$ obtained by partial tracing a random pure state on $\cH \otimes \C^s$, we consider the question whether it is typically separable or typically entangled. For this problem, we show the existence of a sharp threshold $s_0=s_0(d)$ of order roughly $d^3$. More precisely, for any $\e>0$ and for $d$ large enough, such a random state is entangled with very large probability when $s \leq (1-\e)s_0$, and separable with very large probability when $s \geq (1+\e) s_0$. One consequence of this result is as follows: for a system of $N$ identical particles in a random pure state, there is a threshold $k_0 = k_0(N)\sim N/5$ such that two subsystems of $k$ particles each typically share entanglement if $k>k_0$, and typically do not share entanglement if $k<k_0$. Our methods work also for multipartite systems and for ``unbalanced'' systems such as $\C^{d_1} \otimes \C^{d_2} $, ${d_1} \neq {d_2} $. The arguments rely on random matrices, classical convexity, high-dimensional probability and geometry of Banach spaces; some of the auxiliary results may be of reference value.