We study the limiting probability distribution of the homogenization error for second order elliptic equations in divergence form with highly oscillatory periodic conductivity coefficients and highly oscillatory stochastic potential. The effective conductivity coefficients are the same as those of the standard periodic homogenization, and the effective potential is given by the mean. We show that the limiting distribution of the random part of the homogenization error, as random elements in proper Hilbert spaces, is Gaussian and can be characterized by the homogenized Green's function, the homogenized solution and the statistics of the random potential. This generalizes previous results in the setting with slowly varying diffusion coefficients, and the current setting with fast oscillations in the differential operator requires new methods to prove compactness of the probability distributions of the random fluctuation.