This paper studies the sparsistency and rates of convergence for estimating sparse covariance and precision matrices based on penalized likelihood with nonconvex penalty functions. Here, sparsistency refers to the property that all parameters that are zero are actually estimated as zero with probability tending to one. Depending on the case of applications, sparsity priori may occur on the covariance matrix, its inverse or its Cholesky decomposition. We study these three sparsity exploration problems under a unified framework with a general penalty function. We show that the rates of convergence for these problems under the Frobenius norm are of order (s n log p n/n) 1/2, where s n is the number of nonzero elements, p n is the size of the covariance matrix and n is the sample size. This explicitly spells out the contribution of high-dimensionality is merely of a logarithmic factor. The conditions on the rate with which

Recent proposals for implementation of kernel-based nonparametric curve estimators are seen to be faster than naive direct implementations by factors up into the hundreds. The main ideas behind the two different approaches are made clear. Careful speed comparisons in a variety of settings and using a variety of machines and software are done. Various issues on computational accuracy and stability are also discussed. Our speed tests show that the fast methods are as fast or somewhat faster than methods traditionally considered very fast, such as LOWESS and smoothing splines.

We provide a new insight of the difficulty of nonparametric estimation of a whole function. A new method is invented for finding a minimax lower bound of globally estimating a function. The idea is to adjust automatically the direction to the nearly hardest I-dimensional subproblem at each location, and to use locally the difficulty of I-dimensional subproblem. In a variety of contexts, our method can give not only attainable global rates, but also constant factors. Comparing with the existing techniques, our method has the advantages of being easily implemented and understood, and can give constant factors as well.

Variable selection in high-dimensional space characterizes many contemporary problems in scientific discovery and decision making. Many frequently-used techniques are based on independence screening; examples include correlation ranking (Fan & Lv, 2008) or feature selection using a two-sample t-test in high-dimensional classification (Tibshirani et al., 2003). Within the context of the linear model, Fan & Lv (2008) showed that this simple correlation ranking possesses a sure independence screening property under certain conditions and that its revision, called iteratively sure independent screening (ISIS), is needed when the features are marginally unrelated but jointly related to the response variable. In this paper, we extend ISIS, without explicit definition of residuals, to a general pseudo-likelihood framework, which includes generalized linear models as a special case. Even in the least-squares setting, the new method improves ISIS by allowing feature deletion in the iterative process. Our technique allows us to select important features in high-dimensional classification where the popularly used two-sample t-method fails. A new technique is introduced to reduce the false selection rate in the feature screening stage. Several simulated and two real data examples are presented to illustrate the methodology.

Time- and state-domain methods are two common approaches to nonparametric prediction. Whereas the former uses data predominantly from recent history, the latter relies mainly on historical information. Combining these two pieces of valuable information is an interesting challenge in statistics. We surmount this problem by dynamically integrating information from both the time and state domains. The estimators from these two domains are optimally combined based on a data-driven weighting strategy, which provides a more efficient estimator of volatility. Asymptotic normality is separately established for the time domain, the state domain, and the integrated estimators. By comparing the efficiency of the estimators, we demonstrate that the proposed integrated estimator uniformly dominates the other two estimators. The proposed dynamic integration approach is also applicable to other estimation problems in time