In this paper, we propose a dynamically low-dimensional approximation method to solve a class of time-dependent multiscale stochastic diffusion equations. In Cheng etal. (2013) a dynamically bi-orthogonal (DyBO) method was developed to explore low-dimensional structures of stochastic partial differential equations (SPDEs) and solve them efficiently. However, when the SPDEs have multiscale features in physical space, the original DyBO method becomes expensive. To address this issue, we construct multiscale basis functions within the framework of generalized multiscale finite element method (GMsFEM) for dimension reduction in the physical space. To further improve the accuracy, we also perform online procedure to construct online adaptive basis functions. In the stochastic space, we use the generalized polynomial chaos (gPC) basis functions to represent the stochastic part of the solutions. Numerical