The hot flow behaviors, microstructure evolution and fractographs were studied to optimize the hot forming process of the TA32 titanium alloy thin-walled part. A set of microstructure-based constitutive equations were developed based on the experimental data, which described the relationships among the hot flow stresses and the evolution of phase volume fraction, dislocation density, grain size and damage. The constitutive model was imported into ABAQUS 6.14 to simulate the hot forming process for a typical thin-walled part. The effective strain, dislocation density and damage distribution as well as forming defects of formed parts under different process parameters were predicted. A qualified part without wrinkling and fracture defects was produced at a loading speed of 5 mm/s at 800 °C by the modified blank shape, where the maximum damage value was only 18.3%. The accuracy of constitutive model and finite element (FE) simulation was verified by the microhardness tests, which indicates that the FE model based on physical internal-state variables can well optimize the hot forming process of TA32 titanium alloy complex parts.