It is very important to detect the surface defects of the high-speed rail for security concerns. A multimode detecting method, which integrates high resolution of optical image, high precision of photoacoustic detection and strong penetration of ultrasound detecting, is proposed for the rail defect detection. Utilizing the surface defect characteristics obtained from optical signal, the photoacoustic and ultrasound scanning region could be determined, and rail shallow and internal defect characteristics can be acquired subsequently. Eventually, fusing three modal signals mentioned above, the information of the entire rail defect, including type, extension trend and depth can be detected. It has been proved that the multimode method can improve the detecting efficiency, and enlarge the detection range in the meantime.