Many data mining and statistical machine learning algorithms have been developed to select a subset of covariates to associate with a response variable. Spurious discoveries can easily arise in high-dimensional data analysis due to enormous possibilities of such selections. How can we know statistically our discoveries better than those by chance? In this paper, we define a measure of goodness of spurious fit, which shows how good a response variable can be fitted by an optimally selected subset of covariates under the null model, and propose a simple and effective LAMM algorithm to compute it. It coincides with the maximum spurious correlation for linear models and can be regarded as a generalized maximum spurious correlation. We derive the asymptotic distribution of such goodness of spurious fit for generalized linear models and L1 regression. Such an asymptotic distribution depends on the sample size, ambient dimension, the number of variables used in the fit, and the covariance information. It can be consistently estimated by multiplier bootstrapping and used as a benchmark to guard against spurious discoveries. It can also be applied to model selection, which considers only candidate models with goodness of fits better than those by spurious fits. The theory and method are convincingly illustrated by simulated examples and an application to the binary outcomes from German Neuroblastoma Trials.

Askewing'method is shown to effectively reduce the order of bias of locally parametric estimators, and at the same time retain positivity properties. The technique involves first calculating the usual locally parametric approximation in the neighbourhood of a point x'that is a short distance from the place x where the we wish to estimate the density, and then evaluating this approximation at x. By way of comparison, the usual locally parametric approach takes x'= x. In our construction, x'-x depends in a very simple way on the bandwidth and the kernel, and not at all on the unknown density. Using skewing in this simple form reduces the order of bias from the square to the cube of bandwidth; and taking the average of two estimators computed in this way further reduces bias, to the fourth power of bandwidth. On the other hand, variance increases only by at most a moderate constant factor.

This paper studies a weighted local linear regression smoother for longitudinal/clustered data, which takes a form similar to the classical weighted least squares estimate. As a hybrid of the methods of Chen and Jin (2005) and Wang (2003), the proposed local linear smoother maintains the advantages of both methods in computational and theoretical simplicity, variance minimization and bias reduction. Moreover, the proposed smoother is optimal in the sense that it attains linear minimax efficiency when the within-cluster correlation is correctly specified. In the special case that the joint density of covariates in a cluster exists and is continuous, any working within-cluster correlation would lead to linear minimax efficiency for the proposed method.

Factor modeling is an essential tool for exploring intrinsic dependence structures among high-dimensional random variables. Much progress has been made for estimating the covariance matrix from a high-dimensional factor model. However, the blessing of dimensionality has not yet been fully embraced in the literature: much of the available data are often ignored in constructing covariance matrix estimates. If our goal is to accurately estimate a covariance matrix of a set of targeted variables, shall we employ additional data, which are beyond the variables of interest, in the estimation? In this article, we provide sufficient conditions for an affirmative answer, and further quantify its gain in terms of Fisher information and convergence rate. In fact, even an oracle-like result (as if all the factors were known) can be achieved when a sufficiently large number of variables is used. The idea of using data as much as possible

We provide a new insight of the difficulty of nonparametric estimation of a whole function. A new method is invented for finding a minimax lower bound of globally estimating a function. The idea is to adjust automatically the direction to the nearly hardest I-dimensional subproblem at each location, and to use locally the difficulty of I-dimensional subproblem. In a variety of contexts, our method can give not only attainable global rates, but also constant factors. Comparing with the existing techniques, our method has the advantages of being easily implemented and understood, and can give constant factors as well.

An effective bandwidth selection method for local linear regression is proposed in Fan and Gijbels [1995, J. Roy. Statist. Soc. Ser. B, 57, 371394]. The method is based on the idea of the pre-asymptotic substitution and has been tested extensively. This paper investigates the rate of convergence of this method. In particular, we show that the relative rate of convergence is of order <i>n</i>^2/7 if the locally cubic fitting is used in the pilot stage, and the rate of convergence is <i>n</i>^2/5 when the local polynomial of degree 5 is used in the pilot fitting. The study also reveals a marked difference between the bandwidth selection for nonparametric regression and that for density estimation: The plug-in approach for the latter case can admit the root-<i>n</i> rate of convergence while for the former case the best rate is of order <i>n</i>^2/5.

Discrepancy is a kind of important measure used in experimental design. Recently, a so-called discrete discrepancy has been applied to evaluate the uniformity of factorial designs. In this paper, we review some recent advances on application of the discrete discrepancy to several common experimental designs and summarize some important results.

Financial econometrics is an interdisciplinary subject that uses statistical methods and economic theory to address a variety of quantitative problems in finance. This compact, master's-level textbook focuses on methodology and includes real financial data illustrations throughout. The mathematical level is purposely kept moderate, allowing the power of the quantitative methods to be understood without too much technical detail. Wherever possible, the authors indicate where to find the relevant R codes to implement the various methods. This book grew out of a course at Princeton University which is one of the world's flagship programs in computational finance and financial engineering. It will therefore be useful for those with an economics and finance background who are looking to sharpen their quantitative skills, and also for those with strong quantitative skills who want to learn how to apply them to finance.

In the event-related functional magnetic resonance imaging (fMRI) data analysis, there is an extensive interest in accurately and robustly estimating the hemodynamic response function (HRF) and its associated statistics (eg, the magnitude and duration of the activation). Most methods to date are developed in the time domain and they have utilized almost exclusively the temporal information of fMRI data without accounting for the spatial information. The aim of this paper is to develop a multiscale adaptive smoothing model (MASM) in the frequency domain by integrating the spatial and temporal information to adaptively and accurately estimate HRFs pertaining to each stimulus sequence across all voxels in a three-dimensional (3D) volume. We use two sets of simulation studies and a real data set to examine the finite sample performance of MASM in estimating HRFs. Our real and simulated data analyses confirm

Large-scale multiple testing with correlated and heavy-tailed data arises in a wide range of research areas from genomics, medical imaging to finance. Conventional methods for estimating the false discovery proportion (FDP) often ignore the effect of heavy-tailedness and the dependence structure among test statistics, and thus may lead to inefficient or even inconsistent estimation. Also, the commonly imposed joint normality assumption is arguably too stringent for many applications. To address these challenges, in this article we propose a factor-adjusted robust multiple testing (FarmTest) procedure for large-scale simultaneous inference with control of the FDP. We demonstrate that robust factor adjustments are extremely important in both controlling the FDP and improving the power. We identify general conditions under which the proposed method produces consistent estimate of the FDP. As a byproduct that is

Maximum likelihood ratio theory contributes tremendous success to parametric inferences, due to the fundamental theory of Wilks (1938). Yet, there is no general applicable approach for nonparametric inferences based on function estimation. Maximum likelihood ratio test statistics in general may not exist in nonparametric function estimation setting. Even if they exist, they are hard to nd and can not be optimal as shown in this paper. In this paper, we introduce the sieve likelihood statistics to overcome the drawbacks of nonparametric maximum likelihood ratio statistics. New Wilks' phenomenon is unveiled. We demonstrate that the sieve likelihood statistics are asymptotically distribution free and follow 2-distributions under null hypotheses for a number of useful hypotheses and a variety of useful models including Gaussian white noise models, nonparametric regression models, varying coefficient models and generalized varying coefficient models. We further demonstrate that sieve likelihood ratio statistics are asymptotically optimal in the sense that they achieve optimal rates of convergence given by Ingster (1993). They can even be adaptively optimal in the sense of Spokoiny (1996) by using a simple choice of adaptive smoothing parameter. Our work indicates that the sieve likelihood ratio statistics are indeed general and powerful for nonparametric inferences based on function estimation.

In this chapter, we give a selective review of the nonparametric modeling methods using Cox's type of models in survival analysis. We first introduce Cox's model (Cox 1972) and then study its variants in the direction of smoothing. The model fitting, variable selection, and hypothesis testing problems are addressed. A number of topics worthy of further study are given throughout this chapter.

In this paper, we study robust covariance estimation under the approximate factor model with observed factors. We propose a novel framework to first estimate the initial joint covariance matrix of the observed data and the factors, and then use it to recover the covariance matrix of the observed data. We prove that once the initial matrix estimator is good enough to maintain the element-wise optimal rate, the whole procedure will generate an estimated covariance with desired properties. For data with bounded fourth moments, we propose to use adaptive Huber loss minimization to give the initial joint covariance estimation. This approach is applicable to a much wider class of distributions, beyond sub-Gaussian and elliptical distributions. We also present an asymptotic result for adaptive Hubers M-estimator with a diverging parameter. The conclusions are demonstrated by extensive simulations and real data analysis.

We present a new estimator for precision matrix in high dimensional Gaussian graphical models. At the core of the proposed estimator is a collection of node-wise linear regression with nonconvex penalty. In contrast to existing estimators for Gaussian graphical models with O (s log d/n) estimation error bound in terms of spectral norm, where s is the maximum degree of a graph, the proposed estimator could attain O (s/ n+ log d/n) spectral norm based convergence rate in the best case, and it is no worse than exiting estimators in general. In addition, our proposed estimator enjoys the oracle property under a milder condition than existing estimators. We show through extensive experiments on both synthetic and real datasets that our estimator outperforms the state-of-the-art estimators.

Motivated by normalizing DNA microarray data and by predicting the interest rates, we explore nonparametric estimation of additive models with highly correlated covariates. We introduce two novel approaches for estimating the additive components, integration estimation and pooled backfitting estimation. The former is designed for highly correlated covariates, and the latter is useful for nonhighly correlated covariates. Asymptotic normalities of the proposed estimators are established. Simulations are conducted to demonstrate finite sample behaviors of the proposed estimators, and real data examples are given to illustrate the value of the methodology.

High-dimensional statistical tests often ignore correlations to gain simplicity and stability leading to null distributions that depend on functionals of correlation matrices such as their Frobenius norm and other r norms. Motivated by the computation of critical values of such tests, we investigate the difficulty of estimation the functionals of sparse correlation matrices. Specifically, we show that simple plug-in procedures based on thresholded estimators of correlation matrices are sparsity-adaptive and minimax optimal over a large class of correlation matrices. Akin to previous results on functional estimation, the minimax rates exhibit an elbow phenomenon. Our results are further illustrated in simulated data as well as an empirical study of data arising in financial econometrics.

Kernel density estimates are frequently used, based on a second order kernel. Thus, the bias inherent to the estimates has an order of O(<i>h</i>²_<i>n</i>). In this note, a method of correcting the bias in the kernel density estimates is provided, which reduces the bias to a smaller order. Effectively, this method produces a higher order kernel based on a second order kernel. For a kernel function <i>K</i>, the functions W_k(x)=^k-1₁₌₀(^k_l+1)x^lK^(l)(x)/l! and [1/<i>K</i>^{(<i>k</i> 1)}(<i>x</i>)/<i>x</i> d <i>x</i>]<i>K</i>^{(<i>k</i> 1)}(<i>x</i>)/<i>x</i> are kernels of order <i>k</i>, under some mild conditions.

We develop a specification test for the transition density of a discretely-sampled continuous-time diffusion process, based on a comparison of a nonparametric estimate of the transition density or distribution function to their corresponding parametric counterparts assumed by the null hypothesis. We provide a direct comparison of the two densities for an arbitrary specification of the null parametric model using three different discrepancy measures between the null and alternative transition density and distribution functions. We establish the asymptotic null distributions of proposed test statistics and compute their power functions. The finite sample properties are critically investigated via simulation studies and are compared with other tests. Our approaches are illustrated by applications to implied volatility data and Treasury bill data.

An aggregated method of nonparametric estimators based on time-domain and state-domain estimators is proposed and studied. To attenuate the <i>curse of dimensionality</i>, we propose a factor modeling strategy. We first investigate the asymptotic behavior of nonparametric estimators of the volatility matrix in the time domain and in the state domain. Asymptotic normality is separately established for nonparametric estimators in the time domain and state domain. These two estimators are asymptotically independent. Hence, they can be combined, through a dynamic weighting scheme, to improve the efficiency of volatility matrix estimation. The optimal dynamic weights are derived, and it is shown that the aggregated estimator uniformly dominates volatility matrix estimators using time-domain or state-domain smoothing alone. A simulation study, based on an essentially affine model for the term structure, is

This paper is concerned with the problem of top- K ranking from pairwise comparisons. Given a collection of K items and a few pairwise comparisons across them, one wishes to identify the set of K items that receive the highest ranks. To tackle this problem, we adopt the logistic parametric model---the Bradley-Terry-Luce model, where each item is assigned a latent preference score, and where the outcome of each pairwise comparison depends solely on the relative scores of the two items involved. Recent works have made significant progress towards characterizing the performance (eg the mean square error for estimating the scores) of several classical methods, including the spectral method and the maximum likelihood estimator (MLE). However, where they stand regarding top- K ranking remains unsettled.

This paper studies hypothesis testing and parameter estimation in the context of the divide-and-conquer algorithm. In a unified likelihood based framework, we propose new test statistics and point estimators obtained by aggregating various statistics from k subsamples of size n/k, where n is the sample size. In both low dimensional and sparse high dimensional settings, we address the important question of how large k can be, as n grows large, such that the loss of efficiency due to the divide-and-conquer algorithm is negligible. In other words, the resulting estimators have the same inferential efficiencies and estimation rates as an oracle with access to the full sample. Thorough numerical results are provided to back up the theory.

We examine the effect of errors in covariates in nonparametric function estimation. These functions include densities, regressions and conditional quantiles. To estimate these functions, we use the idea of deconvoluting kernels in conjunction with the ordinary kernel methods. We also discuss a new class of function estimators based on local polynomials.

We derive the asymptotic distributions of the spiked eigenvalues and eigenvectors under a generalized and unified asymptotic regime, which takes into account the spike magnitude of leading eigenvalues, sample size, and dimensionality. This new regime allows high dimensionality and diverging eigenvalue spikes and provides new insights into the roles the leading eigenvalues, sample size, and dimensionality play in principal component analysis. The results are proven by a technical device, which swaps the role of rows and columns and converts the high-dimensional problems into low-dimensional ones. Our results are a natural extension of those in Paul (2007) to more general setting with new insights and solve the rates of convergence problems in Shen et al.(2013). They also reveal the biases of the estimation of leading eigenvalues and eigenvectors by using principal component analysis, and lead to a new covariance estimator for the approximate factor model, called shrinkage principal orthogonal complement thresholding (S-POET), that corrects the biases. Our results are successfully applied to outstanding problems in estimation of risks of large portfolios and false discovery proportions for dependent test statistics and are illustrated by simulation studies.

In statistics and machine learning, we are interested in the eigenvectors (or singular vectors) of certain matrices (eg covariance matrices, data matrices, etc). However, those matrices are usually perturbed by noises or statistical errors, either from random sampling or structural patterns. The Davis-Kahan sin theorem is often used to bound the difference between the eigenvectors of a matrix A and those of a perturbed matrix A= A+ E, in terms of l2 norm. In this paper, we prove that when A is a low-rank and incoherent matrix, the l norm perturbation bound of singular vectors (or eigenvectors in the symmetric case) is smaller by a factor of

Over the last two decades, many exciting variable selection methods have been developed for finding a small group of covariates that are associated with the response from a large pool. Can the discoveries by such data mining approaches be spurious due to high dimensionality and limited sample size? Can our fundamental assumptions on exogeneity of covariates needed for such variable selection be validated with the data? To answer these questions, we need to derive the distributions of the maximum spurious correlations given certain number of predictors, namely, the distribution of the correlation of a response variable Y with the best s linear combinations of p covariates X, even when X and Y are independent. When the covariance matrix of X possesses the restricted eigenvalue property, we derive such distributions for both finite s and diverging s, using Gaussian approximation and empirical process