Although depth information of a scene is not stored while taking a picture, some depth information is retained in the captured image. A method is proposed for generating the depth map of a single image based on different scene categories. The image is first classified into a category based on color and texture information, and then the depth map of the image is generated according to the different scene categories. The depth map can be used to generate stereo binocular image pairs by left- and right-shifting the original image. Then, the stereoscopic image with three-dimensional (3-D) visual effect can be viewed through a 3-D stereo display. The experiments showed that the proposed method works well, producing a satisfactory stereoscopic effect.