For any vertex algebra V and any subalgebra A of V, there is a new subalgebra of V known as the commutant of A in V. This construction was introduced by Frenkel-Zhu, and is a generalization of an earlier construction due to Kac-Peterson and Goddard-Kent-Olive known as the coset construction. In this paper, we interpret the commutant as a vertex algebra notion of invariant theory. We present an approach to describing commutant algebras in an appropriate category of vertex algebras by reducing the problem to a question in commutative algebra. We give an interesting example of a Howe pair (ie, a pair of mutual commutants) in the vertex algebra setting.