We use modular invariance to derive constraints on the spectrum of warped conformal field theories (WCFTs) --- nonrelativistic quantum field theories described by a chiral Virasoro and U(1) Kac-Moody algebra. We focus on holographic WCFTs and interpret our results in the simplest holographic set up: three dimensional gravity with Compere-Song-Strominger boundary conditions. Holographic WCFTs feature a negative U(1) level that is responsible for negative norm descendant states. Despite the violation of unitarity we show that the modular bootstrap is still viable provided the (Virasoro-Kac-Moody) primaries carry positive norm. In particular, we show that holographic WCFTs must feature either primary states with negative norm or states with imaginary U(1) charge, the latter of which have a natural holographic interpretation. For large central charge and arbitrary level, we show that the first excited primary state in any WCFT satisfies the Hellerman bound. Moreover, when the level is positive we point out that known bounds for CFTs with internal U(1) symmetries readily apply to unitary WCFTs.