With the rapid development of remote sensing, multiple techniques are now capable of producing digital elevation models (DEMs), such as photogrammetry, Light Detection and Ranging (LiDAR), and interferometric synthetic aperture radar (InSAR). Satellite-derived InSAR DEMs are particularly attractive due to their advantages of large spatial extents, cost-effectiveness, and less dependence on the weather. However, several complex factors may limit the quality of derived DEMs, e.g., the inherited errors may be nonlinear and spatially variable over an entire InSAR pair scene. We propose a segmentation-based coregistration approach for generating accurate InSAR DEMs over large areas. Two matching algorithms, including least squares matching and iterative closest point, are integrated in this approach. Three Advanced Land Observing Satellite Phased Array type L-band Synthetic Aperture Radar (ALOS PALSAR) InSAR DEMs are evaluated, and their root mean square errors (RMSEs) improved from 17.87 to 9.98 m, 51.94 to 15.80 m, and 27.12 to 12.26 m. Compared to applying a single global matching strategy, the segmentation-based strategy further improved the RMSEs of the three DEMs by 3.27, 13.01, and 9.70 m, respectively. The results clearly demonstrate that the segmentation-based coregistration approach is capable of improving the geodetic quality of InSAR DEMs.