In past decade, the increasing popularity of imaging devices, especially smart phones, has led to a great increase in the amount of visual data. The rapidly increasing large scale data pose challenges to the storage and computational resources, and make many computer vision and pattern recognition tasks prohibitively expensive. Dimension reduction techniques explore hidden structures of the original high dimensional data and learn new low dimensional representation to alleviate the challenges. Popular dimension reduction techniques, such as PCA and NMF, do an efficient linear mapping to low dimensional space, while nonlinear techniques overcomes the limitation of linearity at the cost of expensive computational cost (e.g. computing the pairwise distance to find the geodesic distance). In this paper, a piecewise linear dimension reduction technique with global consistency and smoothness constraint is proposed to overcome the restriction of linearity at relatively low cost. Extensive experimental results show that the proposed methods outperform the linear method in the scenario of clustering both consistently and significantly.