We formulate the Riemannian calculus of the probability set embedded with L^2-Wasserstein metric. This is an initial work of transport information geometry. Our investigation starts with the probability simplex (probability manifold) supported on vertices of a finite graph. The main idea is to embed the probability manifold as a submanifold of the positive measure space with a nonlinear metric tensor. Here the nonlinearity comes from the linear weighted Laplacian operator. By this viewpoint, we establish torsion-free Christoffel symbols, Levi-Civita connections, curvature tensors, and volume forms in the probability manifold by Euclidean coordinates. As a consequence, the Jacobi equation, Laplace-Beltrami and Hessian operators on the probability manifold are derived. These geometric computations are also provided in the infinite-dimensional density space (density manifold) supported on a finite-dimensional manifold. In particular, an identity is given connecting the Baker-{É}mery Γ2 operator (carr{é} du champ it{é}r{é}) by connecting Fisher-Rao information metric and optimal transport metric. Several examples are demonstrated.

Real-time crime forecasting is important. However, accurate prediction of when and where the next crime will happen is difficult. No known physical model provides a reasonable approximation to such a complex system. Historical crime data are sparse in both space and time and the signal of interests is weak. In this work, we first present a proper representation of crime data. We then adapt the spatial temporal residual network on the well represented data to predict the distribution of crime in Los Angeles at the scale of hours in neighborhood-sized parcels. These experiments as well as comparisons with several existing approaches to prediction demonstrate the superiority of the proposed model in terms of accuracy. Finally, we present a ternarization technique to address the resource consumption issue for its deployment in real world. This work is an extension of our short conference proceeding paper [Wang et al, Arxiv 1707.03340].

We define the notion of a Ricci curvature lower bound for parametrized statistical models. Following the seminal ideas of Lott–Sturm–Villani, we define this notion based on the geodesic convexity of the Kullback–Leibler divergence in a Wasserstein statistical manifold, that is, a manifold of probability distributions endowed with a Wasserstein metric tensor structure. Within these definitions, which are based on Fisher information matrix and Wasserstein Christoffel symbols, the Ricci curvature is related to both, information geometry and Wasserstein geometry. These definitions allow us to formulate bounds on the convergence rate of Wasserstein gradient flows and information functional inequalities in parameter space. We discuss examples of Ricci curvature lower bounds and convergence rates in exponential family models.

In this paper, we consider the harmonic extension problem, which is widely used in many applications of machine learning. We formulate the harmonic extension as solving a Laplace-Beltrami equation with Dirichlet boundary condition. We use the point integral method (PIM) to solve the Laplace-Beltrami equation. The basic idea of the PIM method is to approximate the Laplace equation using an integral equation, which is easy to be discretized from points. Based on the integral equation, we found that traditional graph Laplacian method (GLM) may fail to approximate the harmonic functions in the classical sense. For the Laplace-Beltrami equation with Dirichlet boundary, we can prove the convergence of the point integral method. The point integral method is also very easy to implement, which only requires a minor modification of the graph Laplacian. One important application of the harmonic extension in machine learning is semi-supervised learning. We run a popular semi-supervised learning algorithm by Zhu et al. over a couple of well-known datasets and compare the performance of the aforementioned approaches. Our experiments show the PIM performs the best. We also apply PIM to an image recovery problem and show it outperforms GLM. Finally, on a model problem of Laplace-Beltrami equation with Dirichlet boundary, we prove the convergence of the point integral method.

The moment-entropy inequality shows that a contin- uous random variable with given second moment and maximal Shannon entropy must be Gaussian. Stam’s inequality shows that a continuous random variable with given Fisher information and minimal Shannon entropy must also be Gaussian. The Crame ́r- Rao inequality is a direct consequence of these two inequalities. In this paper the inequalities above are extended to Renyi entropy, p-th moment, and generalized Fisher information. Gen- eralized Gaussian random densities are introduced and shown to be the extremal densities for the new inequalities. An extension of the Crame ́r–Rao inequality is derived as a consequence of these moment and Fisher information inequalities.

We study the super-resolution (SR) problem of recovering point sources consisting of a collection of isolated and suitably separated spikes from only the low frequency measurements. If the peak separation is above a factor in $(1, 2)$ of the Rayleigh length (physical resolution limit), L1 minimization is guaranteed to recover such sparse signals. However, below such critical length scale, especially the Rayleigh length, the $L_1$ certificate no longer exists. We show several local properties (local minimum, directional stationarity, and sparsity) of the limit points of minimizing two $L_1$ based nonconvex penalties, the difference of $L_1$ and $L_2$ norms $(L_{1-2})$ and capped $L_1 (CL_1)$, subject to the measurement constraints. In one and two dimensional numerical SR examples,the local optimal solutions from dierence of convex function algorithms outperform the global L1 solutions near or below Rayleigh length scales either in the accuracy of ground truth recovery or in nding a sparse solution satisfying the constraints more accurately.

For a random quantum state on $\cH=\C^d \otimes \C^d$ obtained by partial tracing a random pure state on $\cH \otimes \C^s$, we consider the question whether it is typically separable or typically entangled. For this problem, we show the existence of a sharp threshold $s_0=s_0(d)$ of order roughly $d^3$. More precisely, for any $\e>0$ and for $d$ large enough, such a random state is entangled with very large probability when $s \leq (1-\e)s_0$, and separable with very large probability when $s \geq (1+\e) s_0$. One consequence of this result is as follows: for a system of $N$ identical particles in a random pure state, there is a threshold $k_0 = k_0(N)\sim N/5$ such that two subsystems of $k$ particles each typically share entanglement if $k>k_0$, and typically do not share entanglement if $k<k_0$. Our methods work also for multipartite systems and for ``unbalanced'' systems such as $\C^{d_1} \otimes \C^{d_2} $, ${d_1} \neq {d_2} $. The arguments rely on random matrices, classical convexity, high-dimensional probability and geometry of Banach spaces; some of the auxiliary results may be of reference value.

We explain how the classical notions of Fisher information of a random variable and Fisher information matrix of a random vector can be extended to a much broader setting. We also show that Stam’s inequality for Fisher information and Shannon entropy, as well as the more generalized versions proved earlier by the authors, are all special cases of more general sharp inequalities satisfied by random vectors. The extremal random vectors, which we call generalized Gaussians, contain Gaussians as a limiting case but are noteworthy because they are heavy- tailed.

We study the well-posedness of a unified system of coupled forward-backward stochastic differential equations (FB-SDEs) with Levy jumps and double completely-S skew reflections. Owing to the reflections, the solution to an embedded Skorohod problem may be not unique, i.e., bifurcations may occur at reflection boundaries, the well-known contraction mapping approach can not be extended directly to solve our problem. Thus, we develop a weak convergence method to prove the well-posedness of an adapted 6-tuple weak solution in the sense of distribution to the unified system. The proof heavily depends on newly established Malliavin calculus for vector-valued Levy processes together with a generalized linear growth and Lipschitz condition that guarantees the well-posedness of the unified system even under a random environment. Nevertheless, if a more strict boundary condition is imposed, i.e., the spectral radii in certain sense for the reflections are strictly less than the unity, a unique adapted 6-tuple strong solution in the sense of sample pathwise is concerned. In addition, as applications and economical studies of our unified system, we also develop new techniques including deriving a generalized mutual information formula for signal processing over possible non-Gaussian channels with multi-input multi-output (MIMO) antennas and dynamics driven by Levy processes.

The p-th moment matrix is defined for a real random vector, generalizing the classical covariance matrix. Sharp inequalities relating the p-th moment and Renyi entropy are established, generalizing the classical inequality relating the second moment and the Shannon entropy. The extremal distributions for these inequalities are completely characterized.