This dissertation splits into two distinct halves. The first half is devoted to the study of the holography of higher spin gauge theory in AdS_3. We present a conjecture that the holographic dual of W_N minimal model in a 't Hooft-like large N limit is an unusual ``semi-local" higher spin gauge theory on AdS_3×S^1. At each point on the S^1 lives a copy of three-dimensional Vasiliev theory, that contains an infinite tower of higher spin gauge fields coupled to a single massive complex scalar propagating in AdS_3. The Vasiliev theories at different points on the S^1 are correlated only through the AdS_3 boundary conditions on the massive scalars. All but one single tower of higher spin symmetries are broken by the boundary conditions. This conjecture is checked by comparing tree-level two- and three-point functions, and also one-loop partition functions on both side of the duality. The second half focuses on the holography of higher spin gauge theory in AdS_4. We demonstrate that a supersymmetric and parity violating version of Vasiliev's higher spin gauge theory in AdS_4 admits boundary conditions that preserve N=0,1,2,3,4 or 6 supersymmetries. In particular, we argue that the Vasiliev theory with U(M) Chan-Paton and N=6 boundary condition is holographically dual to the 2+1 dimensional U(N)_k×U(M)_{−k} ABJ theory in the limit of large N,k and finite M. In this system all bulk higher spin fields transform in the adjoint of the U(M) gauge group, whose bulk t'Hooft coupling is M/N. Our picture suggests that the supersymmetric Vasiliev theory can be obtained as a limit of type IIA string theory in AdS_4×CP^3, and that the non-Abelian Vasiliev theory at strong bulk 't Hooft coupling smoothly turn into a string field theory. The fundamental string is a singlet bound state of Vasiliev's higher spin particles held together by U(M) gauge interactions.