In this paper, we prove a conjecture of Kottwitz and Rapoport on a union of (generalized) affine Deligne-Lusztig varieties $X(\mu, b)_J$ for a $p$-adic group $G$ and its parahoric subgroup $P_J$. We show that $X(\mu, b)_J \neq \emptyset$ if and only if the group-theoretic version of Mazur's inequality is satisfied. In the process, we obtain a generalization of Grothendieck's conjecture on the closure relation of $\s$-conjugacy classes of a twisted loop group.