Autonomous space object tracking under complex space environment is a popular topic in space engineering research. However, it is a challenging task for measurement equipment, implementing navigation under complex environment, and tracking object with unknown trajectory. An algorithm for space object tracking and azimuth determination using star tracker technology is the first time proposed in this paper. It includes two major steps, star tracking and object tracking. In star tracking stage, a motion-vector algorithm is the first time exploring to track stars in sequence images, which can track stars under complex space environment. With the tracked stars, the star tracker’s attitude can be updated in real-time. In object tracking stage, with the obtained attitude of the star tracker, the Kalman filter (KF) model is built to predict the object state. It takes the measured azimuth as observations rather than the object coordinates in CCD plane, which can avoid the computational complexity due to matrix derivations compared to traditional Extend-Kalman filter, and its convergence rate of the filter is improved consequently. The azimuth and the velocity of the object can be updated by the KF prediction process. In addition, different levels of background noise were added to simulate the complex space environment, and an artificial object is also added in frame with non-linear trajectory in CCD plane. The feasibility of the proposed methods is validated using synthesized sequence images which contain object motions. The simulated results show that the algorithm proposed can track stars and object successfully.