No abstract uploaded!

In statistics and machine learning, we are interested in the eigenvectors (or singular vectors) of certain matrices (eg covariance matrices, data matrices, etc). However, those matrices are usually perturbed by noises or statistical errors, either from random sampling or structural patterns. The Davis-Kahan sin theorem is often used to bound the difference between the eigenvectors of a matrix A and those of a perturbed matrix A= A+ E, in terms of l2 norm. In this paper, we prove that when A is a low-rank and incoherent matrix, the l norm perturbation bound of singular vectors (or eigenvectors in the symmetric case) is smaller by a factor of

In this paper we introduce an appealing nonparametric method for estimating the mean regression function. The proposed method combines the ideas of local linear smoothers and variable bandwidth. Hence, it also inherits the advantages of both approaches. We give expressions for the conditional MSE and MISE of the estimator. Minimization of the MISE leads to an explicit formula for an optimal choice of the variable bandwidth. Moreover, the merits of considering a variable bandwidth are discussed. In addition, we show that the estimator does not have boundary effects, and hence does not require modifications at the boundary. The performance of a corresponding plug-in estimator is investigated. Simulations illustrate the proposed estimation method.

We investigate the asymptotic distributions of coordinates of regression M-estimates in the moderate p/n regime, where the number of covariates p grows proportionally with the sample size n. Under appropriate regularity conditions, we establish the coordinate-wise asymptotic normality of regression M-estimates assuming a fixed-design matrix. Our proof is based on the second-order Poincare inequality (Chatterjee, 2009) and leave-one-out analysis (El Karoui et al., 2011). Some relevant examples are indicated to show that our regularity conditions are satisfied by a broad class of design matrices. We also show a counterexample, namely the ANOVA-type design, to emphasize that the technical assumptions are not just artifacts of the proof. Finally, the numerical experiments confirm and complement our theoretical results.

The central subspace of a pair of random variables $(y,x) \in \mathbb{R}^{p+1}$ is the minimal subspace $\mathcal{S}$ such that $y \perp \hspace{-2mm} \perp x\mid P_{\mathcal{S}}x$. In this paper, we consider the minimax rate of estimating the central space of the multiple index models $y=f(\beta_{1}^{\tau}x,\beta_{2}^{\tau}x,...,\beta_{d}^{\tau}x,\epsilon)$ with at most $s$ active predictors where $x \sim N(0,I_{p})$. We first introduce a large class of models depending on the smallest non-zero eigenvalue $\lambda$ of $var(\mathbb{E}[x|y])$, over which we show that an aggregated estimator based on the SIR procedure converges at rate $d\wedge((sd+s\log(ep/s))/(n\lambda))$. We then show that this rate is optimal in two scenarios: the single index models; and the multiple index models with fixed central dimension $d$ and fixed $\lambda$. By assuming a technical conjecture, we can show that this rate is also optimal for multiple index models with bounded dimension of the central space. We believe that these (conditional) optimal rate results bring us meaningful insights of general SDR problems in high dimensions.