We consider Pólya urns with infinitely many colours that are of a random walk type, in two related versions. We show that the colour distribution a.s., after rescaling, converges to a normal distribution, assuming only second moments on the offset distribution. This improves results by Bandyopadhyay and Thacker (2014–2017; convergence in probability), and Mailler and Marckert (2017; a.s. convergence assuming exponential moment).