Large-scale multiple testing with correlated and heavy-tailed data arises in a wide range of research areas from genomics, medical imaging to finance. Conventional methods for estimating the false discovery proportion (FDP) often ignore the effect of heavy-tailedness and the dependence structure among test statistics, and thus may lead to inefficient or even inconsistent estimation. Also, the commonly imposed joint normality assumption is arguably too stringent for many applications. To address these challenges, in this article we propose a factor-adjusted robust multiple testing (FarmTest) procedure for large-scale simultaneous inference with control of the FDP. We demonstrate that robust factor adjustments are extremely important in both controlling the FDP and improving the power. We identify general conditions under which the proposed method produces consistent estimate of the FDP. As a byproduct that is

When people in a society want to make inference about some parameter, each person would potentially want to use data collected by other people. Information (data) exchange in social contexts is usually costly, so to make sound statistical decisions, people need to compromise between benefits and costs of information acquisition. Conflicts of interests and coordination will arise. Classical statistics does not consider peoples interaction in the data collection process. To address this ignorance, this work explores multi-agent Bayesian inference problems with a game theoretic social network model. Bearing our interest in aggregate inference at the societal level, we propose a new concept finite population learning to address whether with high probability, a large fraction of people can make good inferences. Serving as a foundation, this concept enables us to study the long run trend of aggregate inference quality as population grows.

The multiple testing procedure plays an important role in detecting the presence of spatial signals for large scale imaging data. Typically, the spatial signals are sparse but clustered. This paper provides empirical evidence that for a range of commonly used control levels, the conventional FDR procedure can lack the ability to detect statistical significance, even if the p-values under the true null hypotheses are independent and uniformly distributed; more generally, ignoring the neighboring information of spatially structured data will tend to diminish the detection effectiveness of the FDR procedure. This paper first introduces a scalar quantity to characterize the extent to which the lack of identification phenomenon(LIP) of the FDR procedure occurs. Second, we propose a new multiple comparison procedure, called FDR L, to accommodate the spatial information of neighboring p-values, via a local aggregation of p

Consider a linear model Y= X + z, where X= X n, p and z~ N (0, I n). The vector is unknown and it is of interest to separate its nonzero coordinates from the zero ones (ie, variable selection). Motivated by examples in long-memory time series (Fan and Yao, 2003) and the change-point problem (Bhattacharya, 1994), we are primarily interested in the case where the Gram matrix G= X X is non-sparse but sparsifiable by a finite order linear filter. We focus on the regime where signals are both rare and weak so that successful variable selection is very challenging but is still possible.

Kernel density estimates are frequently used, based on a second order kernel. Thus, the bias inherent to the estimates has an order of O(<i>h</i>²_<i>n</i>). In this note, a method of correcting the bias in the kernel density estimates is provided, which reduces the bias to a smaller order. Effectively, this method produces a higher order kernel based on a second order kernel. For a kernel function <i>K</i>, the functions W_k(x)=^k-1₁₌₀(^k_l+1)x^lK^(l)(x)/l! and [1/<i>K</i>^{(<i>k</i> 1)}(<i>x</i>)/<i>x</i> d <i>x</i>]<i>K</i>^{(<i>k</i> 1)}(<i>x</i>)/<i>x</i> are kernels of order <i>k</i>, under some mild conditions.