Most papers on high-dimensional statistics are based on the assumption that none of the regressors are correlated with the regression error, namely, they are exogeneous. Yet, endogeneity arises easily in high-dimensional regression due to a large pool of regressors and this causes the inconsistency of the penalized least-squares methods and possible false scientific discoveries. A necessary condition for model selection of a very general class of penalized regression methods is given, which allows us to prove formally the inconsistency claim. To cope with the possible endogeneity, we construct a novel penalized focussed generalized method of moments (FGMM) criterion function and offer a new optimization algorithm. The FGMM is not a smooth function. To establish its asymptotic properties, we first study the model selection consistency and an oracle property for a general class of penalized regression methods. These results are then used to show that the FGMM possesses an oracle property even in the presence of endogenous predictors, and that the solution is also near global minimum under the over-identification assumption. Finally, we also show how the semi-parametric efficiency of estimation can be achieved via a two-step approach.

Statistical and machine learning theory has developed several conditions ensuring that popular estimators such as the Lasso or the Dantzig selector perform well in high-dimensional sparse regression, including the restricted eigenvalue, compatibility, and q sensitivity properties. However, some of the central aspects of these conditions are not well understood. For instance, it is unknown if these conditions can be checked efficiently on any given dataset. This is problematic, because they are at the core of the theory of sparse regression. Here we provide a rigorous proof that these conditions are NP-hard to check. This shows that the conditions are computationally infeasible to verify, and raises some questions about their practical applications. However, by taking an average-case perspective instead of the worst-case view of NP-hardness, we show that a particular condition, q sensitivity, has certain

Two measures of sensitivity to initial conditions in nonlinear time series are proposed. The notions give some insight into the relationship between the Fisher information in statistical estimation and initial-value sensitivity in dynamical systems. By using the locally polynomial regression, we develop nonparametric estimates for a conditional density function, its square root and its partial derivatives. The proposed procedures are innovative and of interests in their own right. They are also used to estimate the sensitive measures. The asymptotic normality of the proposed estimators have been proved. We also propose a simple and intuitively appealing method for choosing the bandwidths. Two simulated examples are used as illustrations.

Varyingcoefficient linear models arise from multivariate nonparametric regression, nonlinear time series modelling and forecasting, functional data analysis, longitudinal data analysis and others. It has been a common practice to assume that the varying coefficients are functions of a given variable, which is often called an <i>index</i>. To enlarge the modelling capacity substantially, this paper explores a class of varyingcoefficient linear models in which the index is unknown and is estimated as a linear combination of regressors and/or other variables. We search for the index such that the derived varyingcoefficient model provides the least squares approximation to the underlying unknown multidimensional regression function. The search is implemented through a newly proposed hybrid backfitting algorithm. The core of the algorithm is the alternating iteration between estimating the index through a onestep scheme

We examine the effect of errors in covariates in nonparametric function estimation. These functions include densities, regressions and conditional quantiles. To estimate these functions, we use the idea of deconvoluting kernels in conjunction with the ordinary kernel methods. We also discuss a new class of function estimators based on local polynomials.