The flow field of 3-D rotor-seal system was developed, and the characteristics of compressible gas flied for the system were researched by Computational Fluid Dynamics (CFD) technology. The empirical parameters and experimental parameters of the Muszynska gas exciting force model were further identified by the computational results, which made it more suitable for the rotor-seal system. The nonlinear dynamic analysis of the rotor-seal system applied the power series model of the rotor-seal system obtained by power series expansion. The multiple scale method was used to obtain the bifurcation equations of the rotor-seal system in the 1:2 subharmonic resonance case. Based on the singularity theory and Hopf theory, the 1:2 subharmonic resonance transition sets and the existing condition of the supercritical Hopf bifurcation and subcritical Hopf bifurcation were obtained. The subcritical Hopf bifurcation of the rotor-seal system was inhibited by parameter control to improve the system stability.