The stereoscopic spectral imager combines the space technology and the spectrum technology, which can obtain the three-dimensional information and the spectral information of the target at the same time. A compact stereoscopic spectral imaging system is proposed in this paper. The stereoscopic spectral imaging system works at the wavelength range from 450 to 1000nm. The optical system consists of three telescopes and a spectrometer. The stereoscopic spectral imaging system uses the compact design of the shared spectrometer and detector, which effectively saves the volume weight and the cost of the system, and realizes the requirement of lightweight miniaturization. One of the three telescopes is perpendicular to the earth, the other two observe forward and rear along the direction of the flight. The side angle is 27 degree. The Offner-Chrisp imaging spectrometer based on curved surface prism is used in the system. Curved prism is a special optical element, which can effectively reduce the size of the system, reduce the spectral curvature and improve the performance of the system. The MTF values of all wavelengths at 46lp/mm are greater than 0.6 and the results demonstrated the good performance of the optical design. The stereoscopic spectral imaging system can achieve high-resolution imaging under the condition of wide spectral segment, has the high energy utilization efficiency and can obtain the spectral information in real time.