High throughput genetic sequencing arrays with thousands of measurements per sample and a great amount of related censored clinical data have increased demanding need for better measurement specific model selection. In this paper we establish strong oracle properties of non-concave penalized methods for non-polynomial (NP) dimensional data with censoring in the framework of Coxs proportional hazards model. A class of folded-concave penalties are employed and both LASSO and SCAD are discussed specifically. We unveil the question under which dimensionality and correlation restrictions can an oracle estimator be constructed and grasped. It is demonstrated that non-concave penalties lead to significant reduction of the irrepresentable condition needed for LASSO model selection consistency. The large deviation result for martingales, bearing interests of its own, is developed for characterizing

We provide a new insight of the difficulty of nonparametric estimation of a whole function. A new method is invented for finding a minimax lower bound of globally estimating a function. The idea is to adjust automatically the direction to the nearly hardest I-dimensional subproblem at each location, and to use locally the difficulty of I-dimensional subproblem. In a variety of contexts, our method can give not only attainable global rates, but also constant factors. Comparing with the existing techniques, our method has the advantages of being easily implemented and understood, and can give constant factors as well.

Consider a linear model Y= X + z, where X= X n, p and z~ N (0, I n). The vector is unknown and it is of interest to separate its nonzero coordinates from the zero ones (ie, variable selection). Motivated by examples in long-memory time series (Fan and Yao, 2003) and the change-point problem (Bhattacharya, 1994), we are primarily interested in the case where the Gram matrix G= X X is non-sparse but sparsifiable by a finite order linear filter. We focus on the regime where signals are both rare and weak so that successful variable selection is very challenging but is still possible.

We discuss the difficulties of estimating quadratic functionals based on observations Y (t) from the white noise model Y (t) where Y (t) is a standard Wiener process on Y (t) . The optimal rates of convergence (as Y (t) ) for estimating quadratic functionals under certain geometric constraints are found. Specifically, the optimal rates of estimating Y (t) under hyperrectangular constraints Y (t) and weighted Y (t) -body constraints Y (t) are computed explicitly, where Y (t) is the Y (t) th Fourier-Bessel coefficient of the unknown function Y (t) . We develop lower bounds based on testing two highly composite hypercubes and address their advantages. The attainable lower bounds are found by applying the hardest one-dimensional approach as well as the hypercube method. We demonstrate that for estimating regular quadratic functionals [ie, the functionals which can be estimated

This paper discusses statistical methods for estimating complex correlation structure from large pharmacogenomic datasets. We selectively review several prominent statistical methods for estimating large covariance matrix for understanding correlation structure, inverse covariance matrix for network modeling, large-scale simultaneous tests for selecting significantly differently expressed genes and proteins and genetic markers for complex diseases, and high dimensional variable selection for identifying important molecules for understanding molecule mechanisms in pharmacogenomics. Their applications to gene network estimation and biomarker selection are used to illustrate the methodological power. Several new challenges of Big data analysis, including complex data distribution, missing data, measurement error, spurious correlation, endogeneity, and the need for robust statistical methods, are also discussed.