Detection of the moving targets is a challenging problem in the fields of computer vision especially on complex circumstance. It plays a very important role for the subsequent advanced task such as tracking and behavior understanding are only related to the moving pixels. To well model the moving detection issue, a novel spatial-temporal multi-scale method is proposed to solve the problem of detecting multiple moving objects on complex background in this paper. Moving objects have multi-scale features both in spatial and temporal domain essentially, which means each object has an optimum temporal-spatial detection window. Hence, the problem of detecting moving objects can be transformed into searching optimal spatial-temporal sub-spaces within different scales. A region growing and splitting recursive algorithm in 3D space and an optimal determinant criterion for estimating motion salience and a real time processing architecture are proposed, which can detect multiple objects at different spatial-temporal scales and extract their features on complex background. Experimental results demonstrated that the proposed method is superior to some of the traditional algorithms and works efficiently in detecting multiple moving objects.