The effect of a two-step gradient thermal rolling process on the microstructure evolution and anisotropy as well as the stretch formability of AZ31 magnesium alloys was investigated. Step I was carried out at 300 °C with a small rolling reduction of 15% per pass, while the rolling temperature of Step II was 550 °C with per pass reduction of 40%. Finally, Mg alloy sheets with a thickness of 1 mm were achieved after total four rolling passes. The results indicate that shear bands are generated in rolled samples when the rolling direction is unchanged, while twinning lamellae and recrystallized grains emerge in samples when the rolling direction is changed in Step I. After Step II rolling, the size and amount of shear bands decrease, and grains are refined greatly owing to the enhanced activation of dynamic recrystallization. Besides, non-basal slips, especially prismatic áa？ slips, are found to be promoted as well based on the in-grain misorientation axis (IGMA) analysis. Therefore, improved mechanical properties, reduced anisotropy, and the improvement of stretch formability of AZ31 Mg alloy sheets are achieved.