The leverage effect refers to the generally negative correlation between an asset return and its changes of volatility. A natural estimate consists in using the empirical correlation between the daily returns and the changes of daily volatility estimated from high frequency data. The puzzle lies in the fact that such an intuitively natural estimate yields nearly zero correlation for most assets tested, despite the many economic reasons for expecting the estimated correlation to be negative. To better understand the sources of the puzzle, we analyze the different asymptotic biases that are involved in high frequency estimation of the leverage effect, including biases due to discretization errors, to smoothing errors in estimating spot volatilities, to estimation error, and to market microstructure noise. This decomposition enables us to propose novel bias correction methods for estimating the leverage effect.