In this note we propose a generalization of heterotic/F-theory duality. We introduce a set of non-compact building blocks which we glue together to reach compact examples of generalized duality pairs. The F-theory building blocks consist of non-compact elliptically fibered Calabi-Yau fourfolds which also admit a K3 fibration. The compact elliptic model obtained by gluing need not have a globally defined K3 fibration. By replacing the K3 fiber of each F-theory building block with a T2, we reach building blocks in a heterotic dual vacuum which includes a position dependent dilaton and three-form flux. These building blocks are glued together to reach a heterotic flux background. We argue that in these vacua, the gauge fields of the heterotic string become localized, and remain dynamical even when gravity decouples. This enables a heterotic dual for the hyperflux GUT breaking mechanism which has recently figured prominently in F-theory GUT models. We illustrate our general proposal with some explicit examples.