A new formulation, a gauge formulation of the incompressible Navier-Stokes equations in terms of an auxiliary field $\mathbf{a}$ and a gauge variable $\phi$, $\mathbf{u} =\mathbf{a}+\nabla\phi$, was proposed recently by E and Liu. This paper provides a theoretical analysis of their formulation and verifies the computational advantages. We discuss the implicit gauge method, which uses backward Euler or Crank-Nicolson in time discretization. However, the boundary conditions for the auxiliary field $\mathbf{a}$ are implemented explicitly (vertical extrapolation). The resulting momentum equation is decoupled from the kinematic equation, and the computational cost is reduced to solving a standard heat and Poisson equation. Moreover, such explicit boundary conditions for the auxiliary field $\mathbf{a}$ will be shown to be unconditionally stable for Stokes equations. For the full nonlinear Navier-Stokes equations the time stepping constraint is reduced to the standard CFL constraint ${\triangle t} / {\triangle x} \le C$. We also prove first order convergence of the gauge method when we use MAC grids as our spatial discretization. The optimal error estimate for the velocity field is also obtained.