In this paper, we provide rigorous justification of the hydrostatic approximation and the derivation of primitive equations as the small aspect ratio limit of the incompressible three-dimensional Navier-Stokes equations in the anisotropic horizontal viscosity regime. Setting $\varepsilon >0$ to be the small aspect ratio of the vertical to the horizontal scales of the domain, we investigate the case when the horizontal and vertical viscosities in the incompressible three-dimensional Navier-Stokes equations are of orders $O(1)$ and $O(\varepsilon^\alpha)$, respectively, with $\alpha>2$, for which the limiting system is the primitive equations with only horizontal viscosity as $\varepsilon$ tends to zero. In particular we show that for ``well prepared" initial data the solutions of the scaled incompressible three-dimensional Navier-Stokes equations converge strongly, in any finite interval of time, to the corresponding solutions of the anisotropic primitive equations with only horizontal viscosities, as $\varepsilon$ tends to zero, and that the convergence rate is of order $O\left(\varepsilon^\frac\beta2\right)$, where $\beta=\min\{\alpha-2,2\}$. Note that this result is different from the case $\alpha=2$ studied in [Li, J.; Titi, E.S.: \emph{The primitive equations as the small aspect ratio limit of the Navier-Stokes equations: Rigorous justification of the hydrostatic approximation}, J. Math. Pures Appl., \textbf{124} \rm(2019), 30--58], where the limiting system is the primitive equations with full viscosities and the convergence is globally in time and its rate of order $O\left(\varepsilon\right)$.