31 January 2013 Developing signal processing method for recognizing defects in metal samples based on heat diffusion properties in sonic infrared image sequences
Author Affiliations +
In sonic infrared (SonicIR) imaging, heat is generated in defect areas during the sonic pulse; the heat appears bright in SonicIR images as the indication of a defect. However, in practical applications of SonicIR, there are lots of disturbing bright areas in infrared images, such as heat reflection and paint problem. When crack size is small, the generated heat appears not bright enough to be recognizable. Based on heat diffusion properties in the one-dimensional temporal and two-dimensional spatial domain, a method is developed to automatically recognize defect signals from SonicIR image sequences. The algorithm is verified with the SonicIR image sequences of 100 metal plates which may have different thickness, materials, or crack sizes.



Sonic infrared (SonicIR) is a technology in which a single short pulse of 20 or 40 kHz sound waves passing through a material with a mechanical discontinuity, such as a crack with faying surfaces, will ordinarily cause heating of those surfaces. This heat appears bright in the experimental image as the indication of a defect. The effect appears to begin on a time scale of milliseconds or less after the initiation of a sound pulse in the sample. After that, the image becomes smeared by additional heating and by heat diffusion away from the crack faces. An infrared camera views the location of a crack and records an infrared video, whose total frame number depends on the experimental setup.

Many applications of SonicIR are in the detection of metal fatigue cracks. In those situations, the temperature rise resulting from the heat generated at the crack position can be very small, e.g., less than 0.1 °C. Thus, it can be difficult to identify such a defect in the original infrared image captured at room temperature, as shown in Fig. 1. The adopted method is to observe the images after subtracting a background image taken before the sonic excitation. Such a typical image is shown in Fig. 2. The actual defect is located at the center towards the left as a bright spot indicated in the image. However, there are so many bright areas in Fig. 2, it is difficult to judge, even for an experienced researcher, which area is the actual defect area. Authors have shown two-dimensional (2-D) heat diffusion based support vector machines as an efficient classifier to identify defects in SonicIR image sequences,8 however, it is invalid when the sonic excitation overloads. Thus, we developed an algorithm to automatically recognize the defect area based on heat diffusion properties in time and spatial domains. However, the background may have some points with similar heat diffusion properties as a defect, thus the first step is to locate the sample area and limit further processing in the sample area in order to simplify the processing procedures. The important task of this algorithm is to automatically recognize defects, especially for some weak defect signals which are easily missed by unaided eyes, and remove the disturbing signals, such as bad painting, noise, heat reflection, and other heating sources.

Fig. 1

Original infrared image.


Fig. 2

The infrared image after subtracting an image before the excitation of a sonic pulse.



Subtracted Image Analysis and Sample Area Detection

The experimental setup of SonicIR is shown in Fig. 3. A metal sample is fixed by posts at two ends, a sonic pulse is coupled into the sample at the left side, and an infrared camera captures images at the same side as the sonic gun. One hundred infrared image sequences processed in this paper are from 53 titanium and 47 inconel plates, which have almost the same length and width, however, may have different thickness or materials. The metal samples are painted before the test because of shiny surfaces. There are 13 bad paints among 100 image sequences, which appears to be ‘defect’ as shown in Fig. 2. The experimental setup is the same for all those samples, and also experimental parameters, such as gun pressure, sonic pulse, image size, sampling length, and frequency, are kept the same. Thus, the infrared images of different samples are almost the same as shown in Figs. 1 and 2, which were used in the paper to verify the proposed method because it includes all possible situations happened in the experiment: bright areas at two posts, heat reflection in the background, pure noise, bad painting, and defect. The detailed information about 100 experimental image sequences is listed in Table 1. Disturbing signals including noise, heat generated at two posts, and heat reflection exist in all those image sequences.

Fig. 3

Experimental setup.


Table 1

Sample information.

SampleDefectBad paintDisturbing signal

In SonicIR, heat is generated around the crack area because of friction or clapping, and it is an indication of a defect shown as a bright area. However, there are some other bright areas: the heat generated at two ends because of friction between sample and posts, and the heat reflection in the background or sample edge. The experimental setup was set to capture all those areas, thus the obtained subtracted images among different experiments look similar to Fig. 2. The main difference is the bright area caused by heat reflection, because it may be random. In order to remove the disturbance of the bright areas in the background, the first step of the proposed algorithm is to find the sample area.

The image shown in Fig. 1 has good contrast, so the sample edges could be approximately obtained using some traditional edge algorithms. However, many results have very poor image contrast because of improper focus or environmental disturbances, and the sample edge may be indiscernible. The measured samples are in rectangular shape with almost the same width and height. Then the samples should also have a rectangular shape in the infrared image, which means the distances between edges should be close, and the experimental setup did not change for different samples. Based on those facts, we developed the following steps to extract the sample area:

  • 1. Split the original infrared image T(x,y) into top and bottom parts.

  • 2. Apply a one-dimensional derivative operator to each column of the top and bottom parts



  • 3. Calculate f(x): the position of the maximum absolute value of the derivative, which is shown in Fig. 4 as gray curves.

  • 4. All metal samples have almost the same size of rectangular shape, thus the edges of the metal samples should appear almost continuous in the infrared images. They are almost always fixed horizontally in the experiment, as shown in Fig. 1. The estimated edges should be almost continuous and horizontal, which means it should not have too much deviation from most of the estimated points calculated in step 3. Thus, we can calculate the position value P which got the biggest probability in each edge through a histogram of f(x), then process each edge as follows:


    where P0 is a predefined threshold. Few of the calculated f(x) in step 3 have a large deviation, as shown in Fig. 4 as gray curves. The value g(x) will be further processed. The value chosen for P0 is not critical and adequate edge location is provided by values in the range 5<P0<13.

  • 5. Then, apply curve fitting on g(x) to obtain estimated sample edges g(x) shown in Fig 4 as black curves. Because of the focus problem, sample edges may be a little bit convex in the center, thus the left and right side of each edge were curve fitted separately. Also, the ending points of each gray curve may have a large deviation from the real edge, which are not included when calculating the coefficients of a polynomial. Then, all edge points, including ending points, were replaced by evaluating the obtained polynomial. The detected sample edges were indicated by two white lines, as shown in Fig. 5; they fit quite well with the sample edges.

Fig. 4

Edge positions: the x-axis corresponds to the horizontal direction in Fig. 1 and the y-axis corresponds to the location in the vertical direction in Fig. 1.


Fig. 5

The original infrared image in Fig. 1 with the sample edge labeled. The two white lines are estimated edges in Fig. 4.


The focusing of the infrared camera depends on temperature difference. Sometimes a metal sample and background do not have temperature difference, which results in very poor image contrast. One of such results is shown in Fig. 6, in which two white lines indicate the estimated edges, and the proposed edge detection method can also effectively detect the right edges.

Fig. 6

An infrared image with bad image contrast is labeled with two white lines to indicate edges.



Temperature-Time Plot Analysis

After effectively extracting the sample edges, the following processed results could be limited between two edges. In the sample area, there are only three kinds of temperature-time signals, as shown in Fig. 7, which correspond to noise, defect signal, and bright area signal at two posts. The noise signal corresponds to the dark areas and is purely random; however, it may increase a little bit at a later time because of heat diffusion from hot areas. At both ends of the sample, posts are used to fix the sample in the experiment. During the period of sonic excitation, temperature increases because the post interacts with the sample, and also the sample interacts with the gun tip. Such a curve is labeled the bright area in Fig. 7. The sonic excitation ends at around the 100th frame. Temperature does not drop up to 120th frame because a big amount of energy is created in surrounding areas.

Fig. 7

Temperature-time plots for three different types of points.


Finite element models described in Ref. 9 and analytical models described in Refs. 8 and 10 predict a rise in temperature at defects during the excitation period. In practice, the measured temperature time history is more complicated than the simulations described by the simplified models, which is illustrated by the typical defect signal shown in Fig. 7. However, the feature of the defect signal is that the temperature increases at the sonic excitation and falls once the sonic excitation is removed.

From above analysis, defect temperature-time plots are quite different from noise and other bright areas in the sample area. The following parameter could be used to recognize a defect:


where TInc is the temperature increase, which could be obtained by subtracting the average temperature before the sonic pulse from the average temperature of the central six (60th to 65th) frames during the sonic pulse. TInc is obtained by six frames averaged in this study. It can detect a very weak signal that is easily missed by unaided eyes because averaging can effectively reduce the noise level. The frame range for the temperature increase is not so crucial because most defect signals in this study are similar as shown in Fig. 7 that the temperature increase does not have too much difference during the whole range of ultrasound excitation. TDec is the temperature decrease after sonic excitation, which is obtained by subtracting the six frames’ average temperature after the sonic pulse, which is chosen as the last six frames (115th to 120th frames) as shown in Fig. 7, from the average temperature during the sonic pulse. The frame range for temperature decrease defines the value range of the RT parameter. When it is too close to the ending frame of ultrasound excitation, RT would be very big; if it is too late, RT would be close to 1. TInc of the pure noise signal is very small and almost zero, and RT of bright areas at two ends would be negative or very big. Figure 8 plots the RT coefficients of all 39 defects, which shows that RT of defect signals are almost within the range of 1 to 5. The infrared image sequence is processed by thresholding with the above properties of RT and TInc, the generated image H(x,y) is shown in Fig. 9. Some points in the background may have similar temperature-time plots as a defect; however, those points are easily removed by limiting the result in the sample area obtained in the first step. In the center, there are two bright areas. The left one is the defect, the right one is caused by bad paint, which will be further processed through 2-D heat diffusion property.

Fig. 8

RT coefficients for all 39 defects.


Fig. 9

The result processed by using time domain heat diffusion properties.



Remove Bad Paint Area

Typically, metal samples are painted manually before experiments to increase surface emmisivity and reduce reflection. Thus, some samples may have bad painting effects. Bad paint is mainly caused by loose paint, which is with regular paint thickness as in the rest in the sample. The situation is such that there is bad contact between the paint layer and the metal sample base. Under the excitation of ultrasound, heat is generated at the contacted area between paint and metal due to the bad contact. In addition, since the paint has much lower thermal conductivity, the heat basically is also trapped in the loose paint area since the lateral heat diffusion in the paint is small. All these effects will result in a higher temperature in the paint compared to its surrounding area. Furthermore, bad contact between paint and metal covers a small area, which could be approximately taken as a plane heat source. Thus, the bad paint areas appear to be bright in infrared images, and have similar temperature-time plots as defects in the temporal domain. However, heat is generated between two surfaces of a defect or crack. It could be simply taken as a line heat source from the side of the front surface. The surrounding area absorbs more heat from the defect than the bad paint area. The heat conduction model for bad paint and crack is discussed in Ref. 10, however, the defect plane orientation is different, and bad paint is on the surface. Therefore, a plane heat source and low thermal conductivity for bad paint causes differences in 2-D heat diffusion patterns with a defect.

There are two bright areas as shown in Fig. 8. The first step to remove the bad paint area is to calculate the number of bright areas, which is obtained through the connected component labeling algorithm11 by scanning Fig. 9 pixel-by-pixel to identify 8- or 4-connected pixel regions of adjacent pixels with a gray value of 1. For each connected area, the centroid is obtained by finding the pixel with the peak temperature in the area. The centroid and its corresponding horizontal surrounding values at two different times are plotted in Fig. 10 for the defect area and Fig. 11 for the bad paint area, respectively. In each plot, there are two areas: the dotted black area is the normalized temperature-position plot at an early time, and the dotted white area is the corresponding plot at a later time just before the cease of sonic excitation. Due to the fast diffusion, the area of A in Fig. 10 is not negligible. However, the bad paint area has a slower heat diffusion, area of A in Fig. 11 is negligible compared with area of B. Thus, the following parameter is used to classify defect and paint problem areas:


where A1 and A2 are the area of A and B in Fig. 10. Experimental results with different defects or different bad paint were extracted to calculate RA coefficients through Eq. (4). Figure 12 plots RA coefficients for all 39 defects and 13 bad paints, which shows that RA of the bad paint areas are smaller than that of the defects. A bright area with small RA would be labeled as bad paint and will be removed in the final result, 6% was used for all chosen samples.

Fig. 10

Suppose the peak location of the defect area is (xp, yp), then plot the normalized temperature along the x-direction from xp7 to xp+7. Area B is the plot at an early time and area A is the plot at later time just before the cease of gun excitation.


Fig. 11

The same plot as in Fig. 9 for the paint problem area.


Fig. 12

The RA coefficients comparison between defect and bad paint.


After three main steps, there is only a defect area recognized in the infrared image sequence. Figure 13 shows the image in Fig. 2 by labeling the defect area with red color to indicate the detected defect. It clearly shows that the proposed method can correctly detect the defect area.

Fig. 13

The infrared image in which red color indicates the detected defect.



Performance Evaluation

The proposed method was applied on 100 samples. The recognition results are listed in Table 2. Three defects were missed: one of them was mistaken as noise because of a weak signal, and the others were missed because their RA was less than 6%. Two bad paints were mistaken as defects because their RA was bigger than 6%. Three missed defects plus two bad paints mistaken as defects, and the sample number is 100, thus the overall error is 5%. Disturbing signals in Table 2 include bright patterns in the left and right side of the sample, and the heat reflections in the background. The results indicated that the proposed method is efficient to remove those disturbance patterns.

Table 2

Recognition performance.

Metal defectsBad paintDisturbing signal
Detected as metal defect36/392/130/100
Detected as bad paint2/3911/130/100
Detected as disturbing signal1/390/13100/100
Missed detection3/392/130/100


Discussion and Conclusion

In SonicIR images, background points may have similar heat diffusion properties as defects. The first step of the proposed method is to detect edges of rectangular metal samples and limit further processing in the sample area. The advantage is that it enhanced the detectability of defects with weak signals which are easily missed by unaided eyes, and it can discriminate bad paint areas. At the same time, it does not mistake noise and other disturbing bright areas as a defect. In its present form, it is only valid for rectangular samples. However, the idea of using heat diffusion properties to recognize defects in SonicIR is useful for other practical applications.


This work was supported in part by the Joint Funds of the National Natural Science Foundation of China (No. U1233120) and the National Science Foundation of China (No. 10804078), in part by the Project Foundation of Chongqing Municipal Education Committee (No. KJ100605), and in part by ChongQing Normal University (Nos. 06LB012, 10XLZ009).


1. L. D. Favroet al., “Infrared imaging of defects heated by a sonic pulse,” Rev. Sci. Instrum. 71(6), 2418–2421 (2000).RSINAK0034-6748 Google Scholar

2. X. Hanet al., “Thermosonics: detecting cracks and adhesion defects using ultrasonic excitation and infrared imaging,” J. Adhes. 76(2), 151–162 (2001).JATEE80169-4243 Google Scholar

3. L. D. Favroet al., “Sonic infrared imaging of fatigue cracks,” Int. J. Fatigue 23/1001, 471–476 (2001).IJFADB0142-1123 Google Scholar

4. X. Hanet al., “Recent developments in thermosonic crack detection,” Rev. Prog. Quant. Nondestr. Eval. 21, 552–557 (2002).RPQEDF0743-0760 Google Scholar

5. X. HanL. D. FavroR. L. Thomas, “Recent developments In sonic IR imaging,” Rev. Prog. Quant.e Nondestr. Eval. 22, 500–504 (2003).RPQEDF0743-0760 Google Scholar

6. X. Hanet al., “Acoustic chaos and sonic infrared imaging,” Appl. Phys. Lett. 81, 3188–3190 (2002).APPLAB0003-6951 Google Scholar

7. X. Hanet al., “Acoustic chaos for enhanced detectability of cracks by sonic infrared imaging,” J. Appl. Phys. 95(7), 3792–3797 (2004).JAPIAU0021-8979 Google Scholar

8. Z. Zenget al., “Support vector machines based defect recognition in SonicIR using 2D heat diffusion features,” NDT E Int. 47, 116–123 (2012).NDTIEH0963-8695 http://dx.doi.org/10.1016/j.ndteint.2011.11.001 Google Scholar

9. X. Hanet al., “Finite element modeling of acoustic chaos in sonic infrared imaging,” J. Appl. Phys. 98(1), 014907 (2005).JAPIAU0021-8979 http://dx.doi.org/10.1063/1.1947382 Google Scholar

10. Z. Ouyanget al., “Theoretical modeling of thermosonic imaging of cracks,” Rev. Prog. Quant. Nondestr. Eval. 615, 577–581 (2002).RPQEDF0743-0760 http://dx.doi.org/10.1063/1.1472850 Google Scholar

11. R. C. GonzalezR. E. Woods, Digital Image Processing, 2nd ed., Prentice Hall, Upper Saddle River, NJ (2002). Google Scholar



Zhi Zeng received his bachelor’s degree in optical engineering from the China Institute of Metrology in 1996, master’s degree in optical engineering from Zhejiang University in 2001, and PhD degree in electrical engineering from Wayne State University in 2006. He is currently an associate professor in the Physics Department at Chongqing Normal University since 2006. He has focused his research on infrared imaging NDE technologies.


Ning Tao received her bachelor’s degree in automation from Anhui University in 2000, PhD degree in physics from the University of Science and Technology of China in 2005. She is currently a lecturer in the Physics Department at the Capital Normal University since 2005. She works on infrared imaging NDE technologies.


Lichun Feng received his bachelor’s, Master’s, and PhD degrees in electronics engineering from Tsinghua University in 1997, 2000, and 2005, respectively. He is currently an associate professor in the Physics Department at the Capital Normal University since 2005. He currently works on infrared image processing and thermography NDE technology.


Cunlin Zhang received his bachelor’s degree in physics from Peking University, Master’s degree in physics from the Capital Normal University, and PhD degree in optical engineering from the Beijing Institute of Technology. He is currently a professor in the Physics Department at the Capital Normal University, director of the Beijing Key Lab for Terahertz Spectroscopy and Imaging, director of the Key Laboratory of Terahertz Optoelectronics, Ministry of Education. He has focused his research on developing infrared imaging NDE technologies and terahertz spectroscopy and imaging.


Xiaoyan Han Xiaoyan Han received her bachelors and masters degrees in physics with a specialization in optics from Nankai University, a second masters degree in computer engineering, and a PhD degree in physics from Wayne State University. She is currently a professor in Electrical and Computer Engineering Department at Wayne State University. She has focused her research on developing infrared imaging NDE technologies during the past eighteen years. Her research and development experiences are in fields including optics, electronics, computing, materials, precious measurement instruments, thermal wave imaging, and sonic IR imaging. She has published over 100 scientific papers in journals, conference proceedings and book chapters, and is a coinventor on nine sonic IR Imaging patents.

© The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Zhi Zeng, Zhi Zeng, Ning Tao, Ning Tao, Lichun Feng, Lichun Feng, Cunlin Zhang, Cunlin Zhang, Xiaoyan Han, Xiaoyan Han, } "Developing signal processing method for recognizing defects in metal samples based on heat diffusion properties in sonic infrared image sequences," Optical Engineering 52(6), 061309 (31 January 2013). https://doi.org/10.1117/1.OE.52.6.061309 . Submission:


Back to Top