Imaging spectrometer is widely applied in the field of space remote sensing. Dispersion imaging spectrometer with prism or grating is developed rapidly and used widely. It is developed to the direction of high performance and miniaturization, such as large field of view, high resolution, small volume, etc. For meeting the demand of the development, by comparing the characteristics and the situation of development and application about the two kinds of spectrometer, based on the imaging theory of telecentric optical system, the article studied a design method of prism dispersion imaging spectrometer with telecentric Off-axis Three-Mirror imaging system. The instrument designed by using this method has smaller volume and weight than traditional instrument. It overcomes the biggest defect that traditional prism dispersion imaging spectrometer is bigger, increases its advantages in actual use in contrast to grating dispersion imaging spectrometer, and promotes the development and application of prism dispersion imaging spectrometer.