Magnetic phase diagram and quantum phase transitions in a two-species boson model

We analyze the possible types of ordering in a boson-fermion model. The Hamiltonian is inherently related to the Bose-Hubbard model for vector two-species bosons in optical lattices. We show that such a model can be reduced to the Kugel-Khomskii type spin-pseudospin model, but in contrast to the usual version of the latter model, we are dealing here with the case of spin $S=1$ and pseudospin $1/2$. We show that the interplay of spin and pseudospin degrees of freedom leads to a rather nontrivial magnetic phase diagram including the spin-nematic configurations. Tuning the spin-channel interaction parameter ${U}_s$ gives rise to quantum phase transitions. We find that the ground state of the system always has the pseudospin domain structure. On the other hand, the sign change of ${U}_s$ switches the spin arrangement of the ground state within domains from a ferro- to antiferromagnetic one. Finally, we revisit the spin (pseudospin)-1/2 Kugel-Khomskii model and see the inverse picture of phase transitions.