Concept factorization (CF), as a popular matrix factorization technique, has recently attracted increasing attention in image clustering, due to the strong ability of dimension reduction and data representation. Existing CF variants only consider the local structure of data, but ignore the global structure information embedded in data, which is very crucial for data representation. To address the above issue, we propose an improved CF method, namely local and global regularized concept factorization (LGCF), by considering the local and global structures simultaneously. Specifically, the local geometric structure is depicted in LGCF via a hypergraph, which is capable of precisely capturing high-order geometrical information. In addition, to discover the global structure, we establish an unsupervised discriminant criterion, which characterizes the between-class scatter and the total scatter of the data with the help of latent features in LGCF. For the formulated LGCF, a multiplicative update rule is developed, and the convergence is rigorously proved. Extensive experiments on several real image datasets demonstrate the superiority of the proposed method over the state-of-the-art methods in terms of clustering accuracy and mutual information.