Digital elevation models (DEMs) of buildings in urban areas are becoming increasingly important for a large range of applications, whereas extracting building DEMs over a city is still a complicated and expensive task. In this paper, we present and evaluate two approaches for fast and handy extraction of building DEMs from ALOS PRISM stereo pairs. A Digital Surface Model (DSM) is firstly generated based on the stereo data. Approach 1 uses a digital terrain model (DTM) derived from topographic maps, and Approach 2 creates another DTM by removing buildings on the DSM through a filtering approach. Building DEMs are then acquired by subtracting the two sets of DTMs from the DSM, respectively. The results received with the two approaches are promising. A mean absolute error of 7.07 m and 7.37 m respectively is achieved. Approach 1 has better accuracy in areas with high-density buildings, while Approach 2 has better one in areas with low-density buildings. Combination of the two approaches would be a useful tool for extraction of building DEMs over a city.