Developing signal processing method for recognizing defects in metal samples based on heat diffusion properties in sonic infrared image sequences

Abstract. In sonic infrared (SonicIR) imaging, heat is generated in defect areas during the sonic pulse; the heat appears bright in SonicIR images as the indication of a defect. However, in practical applications of SonicIR, there are lots of disturbing bright areas in infrared images, such as heat reflection and paint problem. When crack size is small, the generated heat appears not bright enough to be recognizable. Based on heat diffusion properties in the one-dimensional temporal and two-dimensional spatial domain, a method is developed to automatically recognize defect signals from SonicIR image sequences. The algorithm is verified with the SonicIR image sequences of 100 metal plates which may have different thickness, materials, or crack sizes.


Introduction
Sonic infrared (SonicIR) is a technology in which a single short pulse of 20 or 40 kHz sound waves passing through a material with a mechanical discontinuity, such as a crack with faying surfaces, will ordinarily cause heating of those surfaces. [1][2][3][4][5][6][7] This heat appears bright in the experimental image as the indication of a defect. The effect appears to begin on a time scale of milliseconds or less after the initiation of a sound pulse in the sample. After that, the image becomes smeared by additional heating and by heat diffusion away from the crack faces. An infrared camera views the location of a crack and records an infrared video, whose total frame number depends on the experimental setup.
Many applications of SonicIR are in the detection of metal fatigue cracks. In those situations, the temperature rise resulting from the heat generated at the crack position can be very small, e.g., less than 0.1°C. Thus, it can be difficult to identify such a defect in the original infrared image captured at room temperature, as shown in Fig. 1. The adopted method is to observe the images after subtracting a background image taken before the sonic excitation. Such a typical image is shown in Fig. 2. The actual defect is located at the center towards the left as a bright spot indicated in the image. However, there are so many bright areas in Fig. 2, it is difficult to judge, even for an experienced researcher, which area is the actual defect area. Authors have shown two-dimensional (2-D) heat diffusion based support vector machines as an efficient classifier to identify defects in SonicIR image sequences, 8 however, it is invalid when the sonic excitation overloads. Thus, we developed an algorithm to automatically recognize the defect area based on heat diffusion properties in time and spatial domains. However, the background may have some points with similar heat diffusion properties as a defect, thus the first step is to locate the sample area and limit further processing in the sample area in order to simplify the processing procedures. The important task of this algorithm is to automatically recognize defects, especially for some weak defect signals which are easily missed by unaided eyes, and remove the disturbing signals, such as bad painting, noise, heat reflection, and other heating sources.

Subtracted Image Analysis and Sample Area
Detection The experimental setup of SonicIR is shown in Fig. 3. A metal sample is fixed by posts at two ends, a sonic pulse is coupled into the sample at the left side, and an infrared camera captures images at the same side as the sonic gun. One hundred infrared image sequences processed in this paper are from 53 titanium and 47 inconel plates, which have almost the same length and width, however, may have different thickness or materials. The metal samples are painted before the test because of shiny surfaces.
There are 13 bad paints among 100 image sequences, which appears to be 'defect' as shown in Fig. 2. The experimental setup is the same for all those samples, and also experimental parameters, such as gun pressure, sonic pulse, image size, sampling length, and frequency, are kept the same. Thus, the infrared images of different samples are almost the same as shown in Figs. 1 and 2, which were used in the paper to verify the proposed method because it includes all possible situations happened in the experiment: bright areas at two posts, heat reflection in the background, pure noise, bad painting, and defect. The detailed information about 100 experimental image sequences is listed in Table 1. Disturbing signals including noise, heat generated at two posts, and heat reflection exist in all those image sequences.
In SonicIR, heat is generated around the crack area because of friction or clapping, and it is an indication of a defect shown as a bright area. However, there are some other bright areas: the heat generated at two ends because of friction between sample and posts, and the heat reflection in the background or sample edge. The experimental setup was set to capture all those areas, thus the obtained subtracted images among different experiments look similar to Fig. 2. The main difference is the bright area caused by heat reflection, because it may be random. In order to remove the disturbance of the bright areas in the background, the first step of the proposed algorithm is to find the sample area.
The image shown in Fig. 1 has good contrast, so the sample edges could be approximately obtained using some traditional edge algorithms. However, many results have very poor image contrast because of improper focus or environmental disturbances, and the sample edge may be indiscernible. The measured samples are in rectangular shape with almost the same width and height. Then the samples should also have a rectangular shape in the infrared image, which means the distances between edges should be close, and the experimental setup did not change for different samples. Based on those facts, we developed the following steps to extract the sample area: 1. Split the original infrared image Tðx; yÞ into top and bottom parts. 2. Apply a one-dimensional derivative operator to each column of the top and bottom parts 3. Calculate fðxÞ: the position of the maximum absolute value of the derivative, which is shown in Fig. 4 as gray curves. 4. All metal samples have almost the same size of rectangular shape, thus the edges of the metal samples should appear almost continuous in the infrared images. They are almost always fixed horizontally in the experiment, as shown in Fig. 1. The estimated edges should be almost continuous and horizontal, which means it should not have too much deviation from most of the estimated points calculated in step 3. Thus, we can calculate the position value P which got the biggest probability in each edge through a histogram of fðxÞ, then process each edge as follows: where P 0 is a predefined threshold. Few of the calculated fðxÞ in step 3 have a large deviation, as shown in Fig. 4 as gray curves. The value gðxÞ will be further processed. The value chosen for P 0 is not critical and adequate edge location is provided by values in the range 5 < P 0 < 13.    Because of the focus problem, sample edges may be a little bit convex in the center, thus the left and right side of each edge were curve fitted separately. Also, the ending points of each gray curve may have a large deviation from the real edge, which are not included when calculating the coefficients of a polynomial. Then, all edge points, including ending points, were replaced by evaluating the obtained polynomial. The detected sample edges were indicated by two white lines, as shown in Fig. 5; they fit quite well with the sample edges.
The focusing of the infrared camera depends on temperature difference. Sometimes a metal sample and background do not have temperature difference, which results in very poor image contrast. One of such results is shown in Fig. 6, in which two white lines indicate the estimated edges, and the proposed edge detection method can also effectively detect the right edges.

Temperature-Time Plot Analysis
After effectively extracting the sample edges, the following processed results could be limited between two edges. In the sample area, there are only three kinds of temperature-time signals, as shown in Fig. 7, which correspond to noise, defect signal, and bright area signal at two posts. The noise signal corresponds to the dark areas and is purely random; however, it may increase a little bit at a later time because of heat diffusion from hot areas. At both ends of the sample, posts are used to fix the sample in the experiment. During the period of sonic excitation, temperature increases because the post interacts with the sample, and also the sample interacts with the gun tip. Such a curve is labeled the bright area in Fig. 7. The sonic excitation ends at around the 100th frame. Temperature does not drop up to 120th frame because a big amount of energy is created in surrounding areas.
Finite element models described in Ref. 9 and analytical models described in Refs. 8 and 10 predict a rise in temperature at defects during the excitation period. In practice, the measured temperature time history is more complicated than the simulations described by the simplified models, which is illustrated by the typical defect signal shown in Fig. 7. However, the feature of the defect signal is that the temperature increases at the sonic excitation and falls once the sonic excitation is removed.
From above analysis, defect temperature-time plots are quite different from noise and other bright areas in the sample area. The following parameter could be used to recognize a defect: where T Inc is the temperature increase, which could be obtained by subtracting the average temperature before the sonic pulse from the average temperature of the central six (60th to 65th) frames during the sonic pulse. T Inc is obtained by six frames averaged in this study. It can detect a very weak signal that is easily missed by unaided eyes because averaging can effectively reduce the noise level. Fig. 4 Edge positions: the x-axis corresponds to the horizontal direction in Fig. 1 and the y -axis corresponds to the location in the vertical direction in Fig. 1. Fig. 5 The original infrared image in Fig. 1 with the sample edge labeled. The two white lines are estimated edges in Fig. 4. The frame range for the temperature increase is not so crucial because most defect signals in this study are similar as shown in Fig. 7 that the temperature increase does not have too much difference during the whole range of ultrasound excitation. T Dec is the temperature decrease after sonic excitation, which is obtained by subtracting the six frames' average temperature after the sonic pulse, which is chosen as the last six frames (115th to 120th frames) as shown in Fig. 7, from the average temperature during the sonic pulse. The frame range for temperature decrease defines the value range of the R T parameter. When it is too close to the ending frame of ultrasound excitation, R T would be very big; if it is too late, R T would be close to 1. T Inc of the pure noise signal is very small and almost zero, and R T of bright areas at two ends would be negative or very big. Figure 8 plots the R T coefficients of all 39 defects, which shows that R T of defect signals are almost within the range of 1 to 5. The infrared image sequence is processed by thresholding with the above properties of R T and T Inc , the generated image Hðx; yÞ is shown in Fig. 9. Some points in the background may have similar temperature-time plots as a defect; however, those points are easily removed by limiting the result in the sample area obtained in the first step. In the center, there are two bright areas. The left one is the defect, the right one is caused by bad paint, which will be further processed through 2-D heat diffusion property.

Remove Bad Paint Area
Typically, metal samples are painted manually before experiments to increase surface emmisivity and reduce reflection. Thus, some samples may have bad painting effects. Bad paint is mainly caused by loose paint, which is with regular paint thickness as in the rest in the sample. The situation is such that there is bad contact between the paint layer and the metal sample base. Under the excitation of ultrasound, heat is generated at the contacted area between paint and metal due to the bad contact. In addition, since the paint has much lower thermal conductivity, the heat basically is also trapped in the loose paint area since the lateral heat diffusion in the paint is small. All these effects will result in a higher temperature in the paint compared to its surrounding area. Furthermore, bad contact between paint and metal covers a small area, which could be approximately taken as a plane heat source. Thus, the bad paint areas appear to be bright in infrared images, and have similar temperaturetime plots as defects in the temporal domain. However, heat is generated between two surfaces of a defect or crack. It could be simply taken as a line heat source from the side of the front surface. The surrounding area absorbs more heat from the defect than the bad paint area. The heat conduction model for bad paint and crack is discussed in Ref. 10, however, the defect plane orientation is different, and bad paint is on the surface. Therefore, a plane heat source and low thermal conductivity for bad paint causes differences in 2-D heat diffusion patterns with a defect.
There are two bright areas as shown in Fig. 8. The first step to remove the bad paint area is to calculate the number of bright areas, which is obtained through the connected component labeling algorithm 11 by scanning Fig. 9 pixelby-pixel to identify 8-or 4-connected pixel regions of adjacent pixels with a gray value of 1. For each connected area, the centroid is obtained by finding the pixel with the peak temperature in the area. The centroid and its corresponding horizontal surrounding values at two different times are plotted in Fig. 10 for the defect area and Fig. 11 for the bad paint area, respectively. In each plot, there are two areas: the dotted black area is the normalized temperature-position plot at an early time, and the dotted white area is the corresponding plot at a later time just before the cease of sonic excitation.   Due to the fast diffusion, the area of A in Fig. 10 is not negligible. However, the bad paint area has a slower heat diffusion, area of A in Fig. 11 is negligible compared with area of B. Thus, the following parameter is used to classify defect and paint problem areas: where A 1 and A 2 are the area of A and B in Fig. 10.
Experimental results with different defects or different bad paint were extracted to calculate R A coefficients through Eq. (4). Figure 12 plots R A coefficients for all 39 defects and 13 bad paints, which shows that R A of the bad paint areas are smaller than that of the defects. A bright area with small R A would be labeled as bad paint and will be removed in the final result, 6% was used for all chosen samples. After three main steps, there is only a defect area recognized in the infrared image sequence. Figure 13 shows the image in Fig. 2 by labeling the defect area with red color to indicate the detected defect. It clearly shows that the proposed method can correctly detect the defect area.

Performance Evaluation
The proposed method was applied on 100 samples. The recognition results are listed in Table 2. Three defects were missed: one of them was mistaken as noise because of a weak signal, and the others were missed because their R A was less than 6%. Two bad paints were mistaken as defects because their R A was bigger than 6%. Three missed defects plus two bad paints mistaken as defects, and the sample number is 100, thus the overall error is 5%. Disturbing signals in Table 2 include bright patterns in the left and right side of the sample, and the heat reflections in the background. The results indicated that the proposed method is efficient to remove those disturbance patterns.

Discussion and Conclusion
In SonicIR images, background points may have similar heat diffusion properties as defects. The first step of the proposed method is to detect edges of rectangular metal samples and limit further processing in the sample area. The advantage is that it enhanced the detectability of defects with weak signals which are easily missed by unaided eyes, and it can discriminate bad paint areas. At the same time, it does not mistake noise and other disturbing bright areas as a defect. In its present form, it is only valid for rectangular samples. However, the idea of using heat diffusion properties to recognize defects in SonicIR is useful for other practical applications.

Acknowledgments
This work was supported in part by the Joint Funds of the National Natural Science Foundation of China (No. U1233120) and the National Science Foundation of China (No. 10804078), in part by the Project Foundation of Fig. 11 The same plot as in Fig. 9 for the paint problem area. Fig. 12 The R A coefficients comparison between defect and bad paint.  Xiaoyan Han Xiaoyan Han received her bachelors and masters degrees in physics with a specialization in optics from Nankai University, a second masters degree in computer engineering, and a PhD degree in physics from Wayne State University. She is currently a professor in Electrical and Computer Engineering Department at Wayne State University. She has focused her research on developing infrared imaging NDE technologies during the past eighteen years. Her research and development experiences are in fields including optics, electronics, computing, materials, precious measurement instruments, thermal wave imaging, and sonic IR imaging. She has published over 100 scientific papers in journals, conference proceedings and book chapters, and is a coinventor on nine sonic IR Imaging patents.