In this paper, we study, in some new ways, the estimation of unimodal densities. Several methods for estimating unimodal densities are proposed: plug-in MLE, pregrouping techniques, linear spline MLE. Based on the maximum likelihood method, an automatic procedure for estimating a unimodal density as well as its mode is proposed. We also give asymptotic theory for the proposed estimators. An important consequence of this study is that having to estimate the location of a mode does not affect the limiting behavior of the proposed unimodal density estimate. Simulation studies illustrate the proposed methods.

In statistics and machine learning, people are often interested in the eigenvectors (or singular vectors) of certain matrices (eg covariance matrices, data matrices, etc). However, those matrices are usually perturbed by noises or statistical errors, either from random sampling or structural patterns. One usually employs Davis-Kahan \sin theorem to bound the difference between the eigenvectors of a matrix \sin and those of a perturbed matrix \sin , in terms of \sin norm. In this paper, we prove that when \sin is a low-rank and incoherent matrix, the \sin norm perturbation bound of singular vectors (or eigenvectors in the symmetric case) is smaller by a factor of \sin or \sin for left and right vectors, where \sin and \sin are the matrix dimensions. The power of this new perturbation result is shown in robust covariance estimation, particularly when random variables have heavy tails. There, we propose new robust covariance estimators and establish their asymptotic properties using the newly developed perturbation bound. Our theoretical results are verified through extensive numerical experiments.

Technological innovations have revolutionized the process of scientific research and knowledge discovery. The availability of massive data and challenges from frontiers of research and development have reshaped statistical thinking, data analysis and theoretical studies. The challenges of high-dimensionality arise in diverse fields of sciences and the humanities, ranging from computational biology and health studies to financial engineering and risk management. In all of these fields, variable selection and feature extraction are crucial for knowledge discovery. We first give a comprehensive overview of statistical challenges with high dimensionality in these diverse disciplines. We then approach the problem of variable selection and feature extraction using a unified framework: penalized likelihood methods. Issues relevant to the choice of penalty functions are addressed. We demonstrate that for a host of statistical problems, as long as the dimensionality is not excessively large, we can estimate the model parameters as well as if the best model is known in advance. The persistence property in risk minimization is also addressed. The applicability of such a theory and method to diverse statistical problems is demonstrated. Other related problems with high-dimensionality are also discussed.

Estimation of longitudinal data covariance structure poses significant challenges because the data usually are collected at irregular time points. A viable semiparametric model for covariance matrixes has been proposed that allows one to estimate the variance function nonparametrically and to estimate the correlation function parametrically by aggregating information from irregular and sparse data points within each subject. But the asymptotic properties of the quasi-maximum likelihood estimator (QMLE) of parameters in the covariance model are largely unknown. We address this problem in the context of more general models for the conditional mean function, including parametric, nonparametric, or semiparametric. We also consider the possibility of rough mean regression function and introduce the difference-based method to reduce biases in the context of varying-coefficient partially linear mean regression

Improving efficiency for regression coefficients and predicting trajectories of individuals are two important aspects in the analysis of longitudinal data. Both involve estimation of the covariance function. Yet challenges arise in estimating the covariance function of longitudinal data collected at irregular time points. A class of semiparametric models for the covariance function by that imposes a parametric correlation structure while allowing a nonparametric variance function is proposed. A kernel estimator for estimating the nonparametric variance function is developed. Two methods for estimating parameters in the correlation structurea quasi-likelihood approach and a minimum generalized variance methodare proposed. A semiparametric varying coefficient partially linear model for longitudinal data is introduced, and an estimation procedure for model coefficients using a profile weighted least squares approach