A unified constitutive model is established to predict the plastic flow behavior of face-centered cubic (FCC) metal in the ultrasonic vibration (UV) assisted forming. The model describes UV softening behavior based on the dislocation dynamics in the UV combined with the stress superposition mechanism. The interaction among dislocation density, grain size and UV is considered, and the residual behavior of UV-assisted forming can be predicted by modelling with dislocation density and grain size as internal state variables. UV-assisted compression tests were performed on oxygen-free high-thermal conductivity (OFHC) copper to verify the predictive ability of the model. The results show that OFHC copper exhibits obvious softening behavior and residual softening behavior in the UV. The model prediction is in good agreement with the experimental result, which effectively captures the softening effect and residual effect of copper.