We show that Hecke algebra of type B and a coideal subalgebra of the type A quantum group satisfy a double centralizer property, generalizing the Schur-Jimbo duality in type A. The quantum group of type A and its coideal subalgebra form a quantum symmetric pair. A new theory of canonical bases arising from quantum symmetric pairs is initiated. It is then applied to formulate and establish for the first time a Kazhdan-Lusztig theory for the BGG category O of the ortho-symplectic Lie superalgebras $\mathfrak{osp}(2m+1|2n)$. In particular, our approach provides a new formulation of the Kazhdan-Lusztig theory for Lie algebras of type B/C.

In this paper, we generalize the categorifical construction of a quantum group and its canonical basis by Lusztig (\cite{Lusztig,Lusztig2}) to the generic form of whole Ringel-Hall algebra. We clarify the explicit relation between the Green formula in \cite{Green} and the restriction functor in \cite{Lusztig2}. By a geometric way to prove Green formula, we show that the Hopf structure of a Ringel-Hall algebra can be categorified under Lusztig's framework.

The first author constructed a q-parameterized spherical category $\sC$ over C(q) in [Liu15], whose simple objects are labelled by all Young diagrams. In this paper, we compute closed-form expressions for the fusion rule of $\sC$, using Littlewood-Richardson coefficients, as well as the characters (including a generating function), using symmetric functions with infinite variables.