Spiral versus modulated collinear phases in the quantum axial next-nearest-neighbor Heisenberg model

Motivated by the discovery of spiral and modulated collinear phases in several magnetic materials, we investigate the magnetic properties of Heisenberg spin $S=1/2$ antiferromagnets in two and three dimensions, with frustration arising from second-neighbor couplings in one axial direction [the axial next-nearest-neighbor Heisenberg (ANNNH) model]. Our results clearly demonstrate the presence of an incommensurate spiral phase at $T=0$ in two dimensions, extending to finite temperatures in three dimensions. The crossover between N\'eel and spiral order occurs at a value of the frustration parameter considerably above the classical value 0.25, a sign of substantial quantum fluctuations. We also investigate a possible modulated collinear phase with a wavelength of four lattice spacings and find that it has substantially higher energy and hence is not realized in the model.