Two fundamental difficulties are encountered in the numerical evaluation of time- dependent layer potentials. One is the quadratic cost of history dependence, which has been successfully addressed by splitting the potentials into two parts—a local part that contains the most recent contributions and a history part that contains the con- tributions from all earlier times. The history part is smooth, easily discretized using high-order quadratures, and straightforward to compute using a variety of fast algo- rithms. The local part, however, involves complicated singularities in the underlying Green’s function. Existing methods, based on exchanging the order of integration in space and time, are able to achieve high-order accuracy, but are limited to the case of stationary boundaries. Here, we present a new quadrature method that leaves the order of integration unchanged, making use of a change of variables that converts the singular integrals with respect to time into smooth ones. We have also derived asymptotic formulas for the local part that lead to fast and accurate hybrid schemes, extending earlier work for scalar heat potentials and applicable to moving boundaries. The performance of the overall scheme is demonstrated via numerical examples.