Aiming at the problems of existing crack extraction algorithms which are difficult to achieve fast and accurate crack extraction of image, an algorithm of crack detection based on Median Filter and Hessian Matrix is proposed. Firstly, median filter of crack gray image in 4 directions, Level, 45 degree, vertical and -45 degree, is conducted, by which noises are removed and roughly extracted crack is obtained. Then according to the Hessian matrix feature of extracting image linear feature, convolution of Differential operation of the Hessian matrix is adopted, and crack is further extracted through eigenvalues response and changing standard deviation of Gaussian function. The proposed algorithm validity is verified by comparison with other crack extraction algorithm. The results show that this algorithm has obvious accuracy rate in crack extraction.