Multiple hypothesis testing is a fundamental problem in high-dimensional inference, with wide applications in many scientific fields. In genome-wide association studies, tens of thousands of tests are performed simultaneously to find if any single-nucleotide polymorphisms (SNPs) are associated with some traits and those tests are correlated. When test statistics are correlated, false discovery control becomes very challenging under arbitrary dependence. In this article, we propose a novel method—based on principal factor approximation—that successfully subtracts the common dependence and weakens significantly the correlation structure, to deal with an arbitrary dependence structure.We derive an approximate expression for false discovery proportion (FDP) in large-scale multiple testing when a common threshold is used and provide a consistent estimate of realized FDP. This result has important applications in controlling false discovery rate and FDP. Our estimate of realized FDP compares favorably with Efron’s approach, as demonstrated in the simulated examples. Our approach is further illustrated by some real data applications. We also propose a dependence-adjusted procedure that is more powerful than the fixed-threshold procedure. Supplementary material for this article is available online.

This paper discusses statistical methods for estimating complex correlation structure from large pharmacogenomic datasets. We selectively review several prominent statistical methods for estimating large covariance matrix for understanding correlation structure, inverse covariance matrix for network modeling, large-scale simultaneous tests for selecting significantly differently expressed genes and proteins and genetic markers for complex diseases, and high dimensional variable selection for identifying important molecules for understanding molecule mechanisms in pharmacogenomics. Their applications to gene network estimation and biomarker selection are used to illustrate the methodological power. Several new challenges of Big data analysis, including complex data distribution, missing data, measurement error, spurious correlation, endogeneity, and the need for robust statistical methods, are also discussed.

This article proposes a consistent and efficient estimator of the high-frequency covariance (quadratic covariation) of two arbitrary assets, observed asynchronously with market microstructure noise. This estimator is built on the marriage of the quasimaximum likelihood estimator of the quadratic variation and the proposed generalized synchronization scheme and thus is not influenced by the Epps effect. Moreover, the estimation procedure is free of tuning parameters or bandwidths and is readily implementable. Monte Carlo simulations show the advantage of this estimator by comparing it with a variety of estimators with specific synchronization methods. The empirical studies of six foreign exchange future contracts illustrate the time-varying correlations of the currencies during the 2008 global financial crisis, demonstrating the similarities and differences in their roles as key currencies in the global market.

In statistics and machine learning, people are often interested in the eigenvectors (or singular vectors) of certain matrices (eg covariance matrices, data matrices, etc). However, those matrices are usually perturbed by noises or statistical errors, either from random sampling or structural patterns. One usually employs Davis-Kahan \sin theorem to bound the difference between the eigenvectors of a matrix \sin and those of a perturbed matrix \sin , in terms of \sin norm. In this paper, we prove that when \sin is a low-rank and incoherent matrix, the \sin norm perturbation bound of singular vectors (or eigenvectors in the symmetric case) is smaller by a factor of \sin or \sin for left and right vectors, where \sin and \sin are the matrix dimensions. The power of this new perturbation result is shown in robust covariance estimation, particularly when random variables have heavy tails. There, we propose new robust covariance estimators and establish their asymptotic properties using the newly developed perturbation bound. Our theoretical results are verified through extensive numerical experiments.

In event studies of capital market efficiency, an earnings surprise has historically been measured by the consensus error, defined as earnings minus the consensus or average of professional forecasts. The rationale is that the consensus is an accurate measure of the markets expectation of earnings. But since forecasts can be biased due to conflicts of interest and some investors can see through these conflicts, this rationale is flawed and the consensus error a biased measure of an earnings surprise. We show that the fraction of forecasts that miss on the same side (FOM), by ignoring the size of the misses, is less sensitive to such bias and a better measure of an earnings surprise. As a result, FOM out-performs the consensus error and its related robust statistics in explaining stock price movements around and subsequent to the announcement date.