We introduce and study mutation of torsion pairs, as a generalization of mutation of cluster tilting objects, rigid objects and maximal rigid objects. It is proved that any mutation of a torsion pair is again a torsion pair. A geometric realization of mutation of torsion pairs in the cluster category of type A_n or A_\infty is given via rotation of Ptolemy diagrams.