F. Labourie [arXiv:1212.5015] characterized the Hitchin components for PSL(n,R) for any n>1 by using the swapping algebra, where the swapping algebra should be understood as a ring equipped with a Poisson bracket. We introduce the rank n swapping algebra, which is the quotient of the swapping algebra by the (n+1)×(n+1) determinant relations. The main results are the well-definedness of the rank n swapping algebra and the "cross-ratio" in its fraction algebra. As a consequence, we use the sub fraction algebra of the rank n swapping algebra generated by these "cross-ratios" to characterize the PSL(n,R) Hitchin component for a fixed n>1. We also show the relation between the rank 2 swapping algebra and the cluster X space.