This paper studies hypothesis testing and parameter estimation in the context of the divide-and-conquer algorithm. In a unified likelihood based framework, we propose new test statistics and point estimators obtained by aggregating various statistics from k subsamples of size n/k, where n is the sample size. In both low dimensional and sparse high dimensional settings, we address the important question of how large k can be, as n grows large, such that the loss of efficiency due to the divide-and-conquer algorithm is negligible. In other words, the resulting estimators have the same inferential efficiencies and estimation rates as an oracle with access to the full sample. Thorough numerical results are provided to back up the theory.

We consider the edge statistics of large dimensional deformed rectangular matrices of the form $Y_t=Y+\sqrt{t}X,$ where $Y$ is a $p \times n$ deterministic signal matrix whose rank is comparable to $n$, $X$ is a $p\times n$ random noise matrix with centered i.i.d. entries with variance $n^{-1}$, and $t>0$ gives the noise level. This model is referred to as the interference-plus-noise matrix in the study of massive multiple-input multiple-output (MIMO) system, which belongs to the category of the so-called signal-plus-noise model. For the case $t=1$, the spectral statistics of this model have been studied to a certain extent in the literature \cite{DOZIER20071099,DOZIER2007678,VLM2012}. In this paper, we study the singular value and singular vector statistics of $Y_t$ around the right-most edge of the singular value spectrum in the harder regime $n^{-2/3}\ll t \ll 1$. This regime is harder than the $t=1$ case, because on one hand, the edge behavior of the empirical spectral distribution (ESD) of $YY^\top$ has a strong effect on the edge statistics of $Y_tY_t^\top$ since $t\ll 1$ is ``small", while on the other hand, the edge statistics of $Y_t$ is also not merely a perturbation of those of $Y$ since $t\gg n^{-2/3}$ is ``large". Under certain regularity assumptions on $Y,$ we prove the edge universality, eigenvalues rigidity and eigenvector delocalization for the matrices $Y_tY_t^\top$ and $Y_t^\top Y_t$. These results can be used to estimate and infer the massive MIMO system. To prove the main results, we analyze the edge behavior of the asymptotic ESD for $Y_tY_t^\top$, and establish some sharp local laws on the resolvent of $Y_tY_t^\top$. These results can be of independent interest, and used as useful inputs for many other problems regarding the spectral statistics of $Y_t$.

No abstract uploaded!

In deconvolution in R^d, d≥1, with mixing density p(∈\mathscr{P}) and kernel h, the mixture density f_p(∈\mathscr{F}_p) is estimated with MDE f_{\hat{p}n}, having upper L1-error rate, an, in probability or in risk; \hat{p}_n∈\mathscr{P}. In one application, \mathscr{P} consists of L1-separable densities in R with differences changing sign at most J times and h(x−y) Totally Positive. When h is known and p is \tilde{q}-smooth, vanishing outside a compact in R^d, plug-in upper bounds are provided for the L2-error rate of \hat{p}_n and its [s]-th mixed partial derivative \hat{p}^{(s)}_n, via ∥f_{\hat{p}_n}−f_p∥_1, with rates (log a^{−1}_n)^{−N_1} and a^{N_2}_n, respectively, for h super-smooth and smooth; \tilde{q}∈R^+, [s] ≤ \tilde{q}, d≥1, N_1>0, N_2>0. For a_n ∼ (logn)^ζ⋅(n^{−δ}), the former rate is optimal for any δ>0 and the latter misses the optimal by the factor (logn)^ξ when δ=.5; ζ>0, ξ>0. N_1 and N_2 appear in optimal rates and lower error and risk bounds in the deconvolution literature.

Frequentists' inference often delivers point estimators associated with condence intervals or sets for parameters of interest. Constructing the condence intervals or sets requires understanding the sampling distributions of the point estimators, which, in many but not all cases, are related to asymptotic Normal distributions ensured by central limit theorems. Although previous literature has established various forms of central limit theorems for statistical inference in super population models, we still need general and convenient forms of central limit theorems for some randomization-based causal analysis of experimental data, where the parameters of interests are functions of a nite population and randomness comes solely from the treatment assignment. We use central limit theorems for sample surveys and rank statistics to establish general forms of the nite population central limit theorems that are particularly useful for proving asymptotic distributions of randomization tests under the sharp null hypothesis of zero individual causal eects, and for obtaining the asymptotic repeated sampling distributions of the causal eect estimators. The new central limit theorems hold for general experimental designs with multiple treatment levels, multiple treatment factors and vector outcomes, and are immediately applicable for studying the asymptotic properties of many methods in causal inference, including instrumental variable, regression adjustment, rerandomization, clustered randomized experiments, and so on. Previously, the asymptotic properties of these problems are often based on heuristic arguments, which in fact rely on general forms of nite population central limit theorems that have not been established before. Our new theorems ll in this gap by providing more solid theoretical foundation for asymptotic randomization-based causal inference.