The statistical structure on a manifold M is predicated upon a special kind of coupling between the Riemannian metric g and a torsion-free affine connection ∇ on the TM, such that ∇g is totally symmetric, forming, by definition, a “Codazzi pair” t∇, gu. In this paper, we first investigate various transformations of affine connections, including additive translation (by an arbitrary (1,2)-tensor K), multiplicative perturbation (through an arbitrary invertible operator L on TM), and conjugation (through a non-degenerate two-form h). We then study the Codazzi coupling of ∇ with h and its coupling with L, and the link between these two couplings. We introduce, as special cases of K-translations, various transformations that generalize traditional projective and dual-projective transformations, and study their commutativity with L-perturbation and h-conjugation transformations. Our derivations allow affine connections to carry torsion, and we investigate conditions under which torsions are preserved by the various transformations mentioned above. While reproducing some known results regarding Codazzi transform, conformal-projective transformation, etc., we extend much of these geometric relations, and hence obtain new geometric insights, for the general case of a non-degenerate two-form h (instead of the symmetric g) and an affine connection with possibly non-vanishing torsion. Our systematic approach establishes a general setting for the study of Information Geometry based on transformations and coupling relations of affine connections.