Transition from half-filled stripe to Néel antiferromagnetism in the t′ -Hubbard model on the honeycomb lattice

We study the ground state of the doped Hubbard model on honeycomb lattice with both nearest $(t)$ and next-nearest-neighbor hoppings $({t}^{\ensuremath{'}})$ in the small doping and strongly interacting region. Previous study on the model without ${t}^{\ensuremath{'}}$ showed the ground state is a half-filled stripe. We employ density matrix renormalization group and extrapolate the results with truncation errors in the converged region. In the ${t}^{\ensuremath{'}}<0$ side, we find the half-filled stripe phase at ${t}^{\ensuremath{'}}=0$ is stable against the frustration of ${t}^{\ensuremath{'}}$ until a critical point $\ensuremath{-}0.4<{t}_c^{\ensuremath{'}}<\ensuremath{-}0.3$, beyond which the ground state switches to antiferromagnetic N\'eel phase with charge modulation. With further increase of ${t}^{\ensuremath{'}}$ to $\ensuremath{-}0.7$, the ground state becomes paramagnetic. In the ${t}^{\ensuremath{'}}>0$ side, the half-filled stripe stretches to ${t}^{\ensuremath{'}}\ensuremath{\approx}0.7$. We do not find obvious enhancement of pairing for the range of ${t}^{\ensuremath{'}}$ studied. We study width-4 cylinders in this paper but the results for spin, charge, and pairing correlation agree qualitatively for periodic and antiperiodic boundary conditions in the half-filled stripe and antiferromagnetic N\'eel phases, suggesting the results are likely to be representative for true two-dimensional systems. The half-filled stripe to antiferromagnetic N\'eel phase transition can be realized on real materials or ultracold atom platform.