We construct a generalization of twistor spaces of hypercomplex manifolds and hyper-Kahler manifolds M, by generalizing the twistor P1 to a more general complex manifold Q. The resulting manifold X is complex if and only if Q admits a holomorphic map to P1. We make branched double covers of these manifolds. Some class of these branched double covers can give rise to non-Kahler Calabi-Yau manifolds. We show that these manifolds X and their branched double covers are non-Kahler. In the cases that Q is a balanced manifold, the resulting manifold X and its special branched double cover have balanced Hermitian metrics.