An algorithm for automatic localization of call numbers embedded in book images is proposed. The algorithm first detects potential character pixels by computing the difference and the position relation between the maximum and minimum gradient in a local window. These pixels are combined to form connected components (CCs). Then, CC-based filtering based on the geometry and shape information is performed to eliminate some nontext components, and the retained components are grouped into text lines candidates. Finally, the call number is selected from the text candidates by using a multilayer perceptron (MLP) classifier. Tests are performed on a set of the images acquired under the different conditions, and experimental results prove that the proposed method is effective.