Ground state phase diagram of the Kitaev-Heisenberg model on a honeycomb-triangular lattice

The Kitaev-Heisenberg model defined on both honeycomb and triangular lattices has been studied intensively in recent years as a possible model to describe spin-orbital physics in iridium oxides. In the model, there are many phases characteristic for each lattice. However, there is no study of how the phases in the two lattices merge with each other when geometry changes from the honeycomb lattice to triangular lattice. We investigate the ground state of the Kitaev-Heisenberg model defined on the system connecting the honeycomb and triangular lattices, named a honeycomb-triangular lattice. We obtain a ground state phase diagram of this model with classical spins by using the Luttinger-Tisza method and classical Monte Carlo simulation. In addition to known phases in the honeycomb and triangular lattices, we find coexisting phases consisting of the known phases. From the exact diagonalization and density-matrix renormalization-group calculations for the model with quantum spins, we find phases similar to the classical results, implying a small effect of quantum fluctuation on the phase diagram.