In order to analyze the chaotic characteristics of the planetary gear system, based on the centralized parameter theory, a torsional vibration model of the planetary gear system is established, which considers nonlinear factors such as time-varying meshing stiffness, backlash and comprehensive meshing error. The Runge-Kutta numerical solution is used to solve the vibration equation. The bifurcation diagram and the largest Lyapunov exponent diagram are used to analyze the bifurcation and chaos characteristics of the system with various parameters. The numerical simulation results show that with the increase of the excitation frequency, the system first enters paroxysmal chaos from periodic motion, and then returns to periodic motion from chaos through inverse doubling bifurcation. The system again enters chaotic motion from periodic motion through jump shock and doubling bifurcation, and finally stabilizes to 1-periodic motion through inverse doubling bifurcation. With the increase of the damping ratio, the system moves from chaotic motion to periodic motion through inverse doubling bifurcation. In the three cases where the amplitude of the integrated meshing error, the backlash, and the stiffness increase respectively, the system moves from periodic motion to chaotic motion by doubling the bifurcation. With the increase of load, the system moves from chaotic motion to periodic motion through jump shock and inverse doubling bifurcation. The above analysis results can provide a theoretical basis for the parameter design of the planetary gear system.