Mars and its natural moon， Phobos， are of great significance for deep space explorations. The N-Body Problem is necessary for the design of orbits around Phobos， since the traditional perturbed Keplerian orbit is proved to be impossible in the vicinity of Phobos. A restricted N-Body Problem model with the ephemeris of celestial bodies is established in Phobos’ body-fixed reference frame. The influencing factors considered in this model include： the polyhedron gravity field of Phobos with the real attitude libration； Mars attractive force with J₂； gravitational perturbations of Sun， Jupiter and Earth. This model is further simplified according to its characteristics with an elliptical restricted three-body problem model considering Phobos’ approximated harmonic libration and gravity harmonics being established. Orbits are simulated with different models with the trajectories and ground tracks being depicted. Comparisons are made among simplified models and the high-fidelity model to confirm the necessity of the proposed models.