1.

Introduction

In any image-processing application, noise filtering is of primary importance. The main objective of noise filtering is the removal of noise from the image. The filtered image constitutes the primary input to high-level vision, which then utilizes domain-specific knowledge to interpret and analyze the image contents. The presence of noise reduces the quality of images and, consequently, can significantly decrease the accuracy of subsequent steps, such as feature extraction or pattern recognition.^{1, 2}

There are many causes for noise in images. The most common is imperfections in image sensors. These produce images that are affected by the corruption caused by photon shot noise and dark-current shot noise.³ As well as this type of noise, transmission through communication channels⁴ or signal representation errors are also quite frequent causes of noise.

In thermographic images, the values of pixels represent temperature levels rather than of luminance levels, as is the case in images taken from the human visible spectrum. Thermographic acquisition devices measure the energy emitted by an object in the infrared spectrum. Using Planck radiation equations, the measured energy is converted to temperature levels using knowledge about the emissivity properties of the object. Noise is present in thermographic images for the same reason outlined above. However, unlike cameras operating in the visible spectrum, thermographic acquisition devices have not been mass produced. This means much lower research investment and, thus, bigger, weightier and more expensive devices of significantly lower quality than that of visible spectrum cameras.⁵ Therefore, noise caused by imperfections in image sensors is much higher in thermographic images. In addition, there is a common cause of noise in thermographic images that does not exist in images taken from the visible spectrum: the variation of emissivity.⁶

Emissivity is a fundamental parameter to carry out the conversion from measured energy to temperature because it defines the capability of a material to absorb and radiate energy. However, it is very difficult to establish the emissivity parameter of an object accurately; normally, the emissivity properties of the object are constant over its entire surface and can change with different temperature levels. The most common approach is to use a rough estimation, which is inevitably translated into noise in the acquired thermographic image. The problem of emissivity variations is even greater when thermographic images are taken in industrial environments where products, such as oil or water, are sprayed on materials such as steel to cool them, altering the emissivity of the surface.

In order to filter noise in images, many different techniques have been developed.⁷ The most commonly used are linear techniques,⁸ which consist of applying an operation through a window $W_x$ , that moves over the image, as can be seen in (1). If all the weighting coefficients, $a_i$ , have the same constant value, the filter is equivalent to replacing the value of a pixel with the average value of the pixels in a surrounding window. This filter is optimal for reducing random white noise.⁹ The reason is that, because the noise trying to be reduced is random, none of the input points are special; each one is just as noisy as its neighbor. Therefore, the optimal choice is to treat them equally,

In order to filter noise while preserving the integrity of edges and detailed information regarding signal changes, much work has been published. ^{11, 12, 13, 14} These papers deal mainly with nonlinear approaches, which provide more satisfactory results than linear techniques. ¹⁵ Among them, some of the most popular filtering schemes are based on orderstatistics,¹⁶ which can take different forms, from the median to the arithmetic mean.

A common drawback of linear and nonlinear techniques is that they require a priori knowledge about the amount of noise in order to make the optimal choice of filtering parameters. Alternatively, an autoadaptive layer, which adapts the performance of the filter based on the properties of the image being processed, can be added to the filter. Examples of adaptive filtering algorithms are the least-mean-square and recursive least-squares approaches.¹⁷ The major inconvenience of these approaches is their slow convergence.¹⁸

Another aspect of filtering noise in images is the temporal constraints of the filtering task; that is, the design of the filtering step when this task is time constrained. This is particularly important when adaptive filters are applied to images, because some adaptive schemes require the execution of computationally demanding algorithms. Existing technology has often turned to simplifications and modality specific optimization in order to sustain the expected performances.

In this work, a real-time adaptive method for noise filtering is proposed. The method is designed to remove noise from a stream of thermographic line scans. However, most of the work presented in this paper is totally applicable to any general linear image-processing application in real time. These processing applications are normally used to inspect a moving object. The filter is divided in two steps: a filtering process within each line scan based on spatial overlapping, and an adaptive filtering process between successive line scans to smooth the final two-dimensional image while preserving the integrity of edges. Both steps are designed to be applied by a low-cost computer in real time with the acquisition process.

This paper is organized as follows: Section 2 describes the acquisition process for thermographic images; Sec. 3 presents the proposed new approaches for noise filtering in this kind of images; Sec. 4 discusses the experimental results by comparing them to average and median filters; Sec. 5 analyzes the real-time performance of the proposed approach, and finally, Sec. 6 reports conclusions.

2.

Acquisition of Thermographic Images

The thermographic images considered in this work are obtained using a high-speed infrared line scanner (IRLS), which is a continuously turning pyrometer. The IRLS measures the energy radiated by the surface of an object. Using Planck radiation equations, the measured energy is converted to an analog signal that represents the temperature. At every turn, the IRLS generates an analog signal that represents the temperature of a measurement line across the object, perpendicular to the direction of movement. Figure 1 shows an overview of the operation of an IRLS.

Fig. 1

Operation of an infrared line scanner.

The analog signal generated by the IRLS at every turn is sampled by a computer, resulting in a line scan, that is, a set of samples that correspond to the temperature measurements of a segment across the object. Figure 2 shows examples of consecutively acquired line scans from a hot steel strip moving forward along a track. In the line scans of Fig. 2, noise can clearly be seen. The origin of this noise is mainly due to the emissivity variations caused by cooling products sprayed on the steel strips during their manufacturing.

Fig. 2

Thermographic line scans taken from a hot steel strip.

Repetitive line scanning and the movement of the object make the acquisition of a rectangular image possible. The image obtained consists of a sequence of line scans. An example can be seen in Fig. 3 , which shows a thermographic image taken from the same hot-steel strip from which the line scans of Fig. 2 were taken. The line scans of Fig. 2 were taken at several positions (shown at the top of each line scan) along the steel strip. The thermographic image in Fig. 3 is a representation of the sequence of acquired line scans, each of them constituting a column of the image, and where the temperature level is represented using a color scale. Images formed as a sequence of line scans can contain information about very long objects. For instance, the thermographic image in Fig. 3 contains 70,000 line scans that represent a steel strip nearly $12\mathrm{km}$ long. More information about the acquisition and the structure of the thermographic images can be found in Ref. 6.

Fig. 3

Thermographic image taken from a hot steel strip.

By using infrared linear scanners rather than an infrared camera that takes two-dimensional (2D) images, extremely long moving objects, such as steel strips, can be inspected effectively. ¹⁹ It is interesting to note that the formation of the image based on a stream of line scans is very similar to the one carried out using linear cameras that take images in the visible spectrum.²⁰ The main difference is that when using the infrared line scanner there is only one temperature sensor, which reduces costs. Furthermore, the analog-to-digital (A/D) conversion is carried out in the computer, while the modern visible spectrum linear cameras have an array of luminance sensors that carry out the A/D conversion, providing a digital signal representing the line scan.

A particular feature of these thermographic line scans is that the number of samples obtained in every line scan depends on the acquisition rate used in the A/D converter installed in the computer. When the acquisition rate of the A/D converter is high, the temperature at a specific position can be measured by several adjacent samples, that is, spatial overlapping occurs. The higher the acquisition rate is, the greater the overlapping is.

2.1.

Noise Filtering

The noise-filtering proposal in this paper consists of the application of two independent filters: a filtering process within each line scan, called intrascan filtering (in the vertical direction of the 2-D image), and an adaptive filtering between successive line scans, called inter-scan filtering (in the horizontal direction of the 2D image). The reason for using an independent filter in each direction rather than a bidimensional filter is that the samples of the 2-D image are not at the same distance in the vertical and horizontal directions. The distance between adjacent samples within the same line scan (vertical direction) is approximately constant, but the distance between adjacent samples in successive line scans (horizontal direction) varies with the speed of the object and can be 30 times greater than the vertical distance.

One of the major issues of any filtering strategy is the decision about what should be filtered and what not. Sometimes, smoothing a signal is effective because it reduces noise; whereas in other cases, this can lead to the loss of valuable information. The strategy proposed in this paper is based on two steps, each of them with a different objective. The filtering strategy is illustrated in Fig. 4 . The acquisition process obtains line scans periodically. Each line scan is filtered independently, and the results are stored in a buffer. This process is called intrascan filtering. The proposed intrascan filtering treats each line scan as an independent entity. Thus, it uses only the information available in a given line scan for its filtering. This makes the detection and removal of temperature changes caused by emissivity variations impossible because these could only be detected by comparing the current line scan to others acquired previously.

Fig. 4

Filtering strategy.

The next step in the proposed filtering strategy is interscan filtering. The objective of interscan filtering is to detect temperature changes (which are noise in the signal) and to remove them. To carry out this task, interscan filtering is applied over several line scans that have been filtered previously using intrascan filtering, in order to obtain a smooth and noise-free signal. Below, a detailed description of both intrascan and interscan filtering is presented.

3.

Intrascan Filtering

During line-scan acquisition, a high acquisition rate in the A/D converter can provoke spatial overlapping of the adjacent samples measured. The higher the acquisition rate is, the greater the overlapping is. Figure 5 shows the result of using different A/D conversion rates. In the case in the center, a medium conversion rate is used that provides a line scan with information about the whole measurement line. In the case on the right, using a high acquisition rate, the temperature of a position is measured in several adjacent samples; that is, spatial overlapping occurs. In the case on the left, when the conversion rate is low, only information about nonadjacent positions of the measurement line is obtained.

Fig. 5

Result of different A/D conversion rate.

A classical approach to filtering noise in a line scan is to carry out the acquisition using a high A/D conversion rate and later apply a linear or nonlinear filter. Next, a decimation process could be used to reduce the number of samples in the line scan in order to save storage space in disk or memory usage. However, to apply this filter, the proper coefficients must be selected. The approach proposed in this paper is to acquire the line scan at the highest possible rate, and then filter this line scan using a set of coefficients calculated automatically based on the spatial overlapping on each position of the measurement line. The result of the proposed filter is a line scan without spatial overlapping that covers the whole measurement line, that is, a line scan similar to the example in the center in Fig. 5, but where each sample is obtained from multiple samples covering the same measured space. For example, in a line scan acquired using a conversion rate double the medium conversion rate in Fig. 5, the measurement space of a sample overlaps the measurement space of adjacent samples. This results in samples that contain temperature information about the measurement space of their previous sample and about the measurement space of their next sample. This information can be used to filter the line scan effectively. The process can be seen in Fig. 6 .

Fig. 6

Intrascan filter based on spatial overlapping.

Every sample in the filtered line scan $y\left[i\right]$ can be calculated using the samples from original line scan $x$ , whose measurement space includes a part of the measurement space in the sample $y\left[i\right]$ , that is, the samples of $x$ that overlap $y\left[i\right]$ (as seen in Fig. 6). This process involves two steps: first, determining the set of samples in line scan $x$ that overlap $y\left[i\right]$ , and second, calculating the weight of each sample of that set in the calculation of the value of $y\left[i\right]$ . This depends on the size of the area intersected of the samples in $x$ with $y\left[i\right]$ , in relation to the size of the sample $y\left[i\right]$ . From this relationship, a ratio is calculated hereafter referred to as the ratio of intersection.

Figure 6 shows a simplified view of the measurement space of each sample. The truth is that the measurement space of a sample is only circular when the target beam (Fig. 1) of the sensor is perpendicular to the object (intersection of a cone with a plane parallel to the cone base, shown in Fig. 7 ), and this only happens when the sample is in the middle of the line scan. In other samples the measurement space is an ellipse (intersection of a cone with a plane not parallel to the cone base, shown in Fig. 7). The further the sample is from the center of the line scan, the longer the major and minor axes of the ellipse are. However, the major semiaxis length increases more than the minor semiaxis length due to the geometry used by the IRLS to measure the temperature. The real spatial overlapping process can be seen in Fig. 8 .

Fig. 7

Measurement space of the samples.

Fig. 8

Measurement space of the samples.

The problem of the model shown in Fig. 8 is not only that the size of the measurement space changes from sample to sample, but that the samples in the line scan are not equidistant; samples near the center of the line scan are closer together than samples near the borders of the line scan.

In order to obtain the ratio of intersection between the original and filtered samples, the position of each original sample and its major and minor semiaxes must be known. This information can be obtained using the procedure presented in Ref. 21. Using this procedure, from a given sample, $J$ , the position of the center of this sample, $d_J$ , and the length of the major semiaxis, $a_J$ , and of the minor semiaxis, $b_J$ , can be calculated. This can be reduced to the geometrical problem of the calculation of the area of intersection of an ellipse and a circle, which can be seen as the calculation of the area of a circular segment of an ellipse plus the area of a circular segment of a circle, as can be seen in Fig. 9 .

Fig. 9

Intersection between samples.

The area of a circular segment can be calculated using ², where $R$ is the radius of the circle and $\theta$ is the angle of the segment. To calculate the area of a circular segment of an ellipse with semiaxes $a_J$ and $b_J$ , the equation is the same but using $a_J$ as $R$ and multiplying the result by the division between minor and major semiaxes $(b_J∕a_J)$ ,

To obtain $R$ and $\theta$ , the points of intersection between the circle and the ellipse must be calculated. This can be done by solving the equation system produced by the equations of those shapes in the space.

The intrascan filtering proposed involves determining which samples in the original line scan provide information about each filtered sample and then using this information to calculate a filtered value for each sample in the filtered line scan. This process can be implemented using three functions:

Using these functions, the filtered value for the sample $i$ can be calculated as follows:

The calculation of FSI, LSI, and AI for each sample $i$ of the filtered line scan is a computationally demanding task. However, these values can be precalculated and stored for later use. FSI and LSI can be stored in two vectors, and AI can be stored in a matrix. In this way, the computation needed to apply the proposed approach is similar to that required by a convolution.

While the line scan is being acquired, the object whose temperature is being measured moves perpendicularly to the scanning trajectory. This means that during the time elapsed between the acquisition of the first and last samples of the line scan, the measured object has moved. Following this reasoning, the proposed model to calculate the ratio of intersection is not completely accurate, because the measurement line is not perpendicular to the movement of the object. However, taking into consideration that the usual acquisition rate of line scans is $\sim 10\mathrm{m}$ and that the maximum speed of the object being measured by these kinds of devices is $\sim 25\mathrm{m}∕\mathrm{s}$ , the offset from the first to the last sample of the line scan is small enough to be considered insignificant for the calculations.

3.1.

Interscan Filtering

The proposed intrascan filtering treats each line scan as an independent entity. Thus, it uses only the information available in one line scan for its filtering, which makes the detection and the removal of random temperature changes caused by emissivity variations impossible. The objective of interscan filtering is to detect temperature changes that are real noise in the signal and to remove them. To carry out this task, interscan filtering is applied over several line scans, previously filtered using the proposed intrascan filtering, in order to obtain a smooth noise-free signal.

Changes in temperature are due to changes in manufacturing conditions over time. For example, in the case of a steel strip, a reduction in the speed of the rolling process or excessive pressure on one part of the strip can produce these changes. A very important aspect of temperature changes is that they always appear in the direction of the movement of the object being inspected, that is, in the horizontal direction of the 2-D image. This can be used by the interscan filtering process to distinguish deterministic temperature changes that must not be removed from those that are considered noise.

The proposed method to detect deterministic temperature changes is by means of boundary detection, which in this case is accomplished through real-time edge detection. Edges mark discontinuities that shall not be removed. On the other hand, changes where edges have not been found indicate noise that must be removed.

The steps for the proposed edge detection are based on the calculation of the gradient in real time. Then, the gradient is projected and thresholded to simplify edge detection. Finally, the position of the edges detected in real time is used to calculate an adaptive average by means of a window whose size grows from one edge to the next. The steps are described below.

3.1.1.

Calculation of the gradient

The gradient of an image is obtained as the result of the convolution of the image with a gradient operator. When choosing a gradient operator, three important parameters must be carefully selected: the direction, the size, and the shape. The fact that the edges will only be searched in the direction of the movement of the object makes the selection of direction and size easier. The direction of the operator will be the same as the movement, that is, horizontal in Fig. 3. Furthermore, the size, normally defined as $A$ rows by $B$ columns, can be simplified as $1\times B$ , since only the modification of the number of columns of the operator $\left(B\right)$ will make significant changes to the resulting gradient due to the searched direction. To simplify the next operations, $N$ , defined as $(B-1)∕2$ , will be used when referring to the operator length.

To calculate the gradient, the difference operator $(-1\dots -101\dots )$ ⁸ is selected. To apply the difference operator (Dif), a convolution operation can be used. However, because the application of the difference operator in one pixel corresponds to the calculation of the difference between two averages, it can also be applied using Eq. ⁴, where $P[i,j]$ is the pixel $j$ of the line scan $i$ ,

Eq. 4

$$\mathrm{Dif}̱\mathrm{Grad}[i,j]=\frac{{\sum }_{k=i+1}^{k=i+N}P[k,j]}{N}-\frac{{\sum }_{k=i-N}^{k=i-1}P[k,j]}{N}.$$

Equation ⁴ can also be seen as ⁵, the difference between the next and previous window (size $N$ ), where $\mathrm{AP}[f,t,j]$ is the average pixel $j$ from the line scan $f$ to $t$ ,

Eq. 5

$$\mathrm{Dif}̱\mathrm{Grad}[i,j]=\mathrm{AP}[i+1,i+N,j]-\mathrm{AP}[i-N,i-1,j].$$

Using the recursive definition of the average, Eq. ⁶ can be proposed for the calculation of AP, which makes it possible to calculate the operator recursively, therefore requiring a low number of operations,

Eq. 6

$$\mathrm{AP}[a,b,j]=\mathrm{AP}[a-1,b-1,j]+\frac{P[b,j]-P[a-1,j]}{b-a+1}.$$

The filtering process must be applied in real time. This means that the time necessary to calculate the gradient in a line scan will be a part of the total time necessary to process a new acquired line scan. Therefore, the best operator for the gradient calculation is the difference operator, which needs far fewer mathematical operations than the convolution.

The calculation of the gradient uses information from line scans acquired previously and from the future. The only possibility to calculate the gradient using a window centered in the current line scan is to delay the calculation until there is enough information to create that window of line scans. This process delays the calculation of the gradient and, consequently, delays the possible detection of an edge. However, this delay $(<1\mathrm{s})$ is insignificant for most applications.

Figure 10 shows the gradient produced by the difference operator (using $N=100$ ) when applied to the thermographic image shown in Fig. 3.

Fig. 10

Gradient produced by the difference operator.

3.1.2.

Projection of the gradient

The next step is the projection of the gradient, which simplifies the thresholding that must be carried out to eliminate noise from the gradient. This projection is carried out using Eq. ⁷, where LSL is the line scan length (number of samples in the line scan). A quadratic projection is used to eliminate the influence of the sign of the gradient in the projection,

Eq. 7

$$\mathrm{GradProj}\left[i\right]=\frac{1}{\mathrm{LSL}}\sum _{j=1}^{\mathrm{LSL}}{\left(\mathrm{Grad}[i,j]\right)}^2.$$

Figure 11 shows the projection of the gradient shown in Fig. 10.

Fig. 11

Projection of the gradient.

3.1.3.

Thresholding of the projection

Once the projection of the gradient is available, it is thresholded. Thresholding is used to differentiate noise from real edges. An edge is found when there are data in the projection over the threshold value, $T$ . When adjacent edges are found (adjacent values of the projection over the threshold), only the edge with the higher value in the projection of all of the adjacent positions will be considered.

Figure 12 shows an example of the thresholding (using $T=25$ ) carried out over the projection of the gradient produced by the difference operator (using $N=100$ ), as shown in Fig. 11. As can be seen, the noise is below the threshold value and edges are obtained from the peaks above it (Fig. 13 ). Only the highest value of each peak will be considered to establish the horizontal position of its corresponding edge. Figure 14 shows the position where edges have been found on the thermographic image.

Fig. 12

Thresholding of the projection.

Fig. 13

Position of the edges as the maximums of every peak.

Fig. 14

Edges in the thermographic image.

3.1.4.

Tuning of the edge-detection parameters

Two parameters in the edge-detection process must be configured: the operator length, $N$ , and the threshold level, $T$ . These parameters determine how the edge-detection process is to distinguish between real temperature changes and noise. Therefore, it is very important that the values assigned to them are correct. The most robust tuning method is by means of a training procedure applied over a set of images whose edges have been tagged. This tuning method involves several important tasks:

• 1 selection of a representative set of images to create the image test set

• 2 tagging the edges of each image in the test set, which creates the ground truth

• 3 selection of a metric that provides information about the accuracy of the edge detection process applied over the test set using specific values of $N$ and $T$

• 4 selection of an optimization algorithm that can find the best values of the two parameters for the detection of edges in the test set.

The first two tasks involve human work. They can be carried out by technicians with knowledge of the images being filtered, and of the expected results. If two or more technicians tag the edges of the images in the test set, the ground truth may be fuzzy. Fuzziness, more than a drawback, is an opportunity for the detection metric and the optimization algorithm to perform better.²²

Edge detection metrics measure the performance of an edge detector when applied over an image by comparing the detection results to its ground truth. Several metrics have been proposed to assess edge detection,²³ such as the probability of error,²⁴ the figure of merit,⁸ the $R$ metric,²⁵ or the Jaccard and Yule metrics.²⁶ Some of the proposed metrics can take uncertainty into account, such as the success segmentation ratio (SSR) metric.²²

Finally, the optimization algorithm can use the edge-detection metric as the feedback for the search for the best values for the configuration parameters. Among these algorithms, support vector machines²⁷ or genetic algorithms²⁸ perform efficiently.

3.1.5.

Adaptive filtering from detected edges

The process described to detect edges in the thermographic image is designed to be applied in realtime, requiring a low number of computational operations. Information about edges in the image can be used to produce a filtered output in realtime. Indeed, changes in the images appear as a consequence of noise or variations of the manufacturing conditions. Because deterministic changes are detected by the proposed edge detection method, others changes can be removed by applying a smoothing filter.

The proposed filter consists of applying an averaging filter, beginning at the next line scan after the last detected edge and finishing at the current line scan. When applying this filter, the averaging window grows as long as the acquired image does not present an edge, that is, the presence of a temperature change due to manufacturing conditions. When an edge is found, the window size for the average calculation is reset to avoid blurring the edges. During the transitory state of the image (that is, when the projection of the gradient is above the threshold), a median filter can be applied to remove the noise of the slope while preserving the sharpness of the temperature change. The median filter is applied recursively to maintain the number of operations as low as possible.

4.

Experimental Results

To test the proposed filtering approach, several thermographic images have been filtered and compared to two well-known filtering methods: the average filter and the median filter. Images have been taken from heated steel strips in an industrial environment. The infrared line scanner used acquired line scans at $100\mathrm{Hz}$ , each of them consisting of 833 samples.

4.2.

Intrascan Filtering Results

Figure 15 shows the results of the proposed intrascan filtering approach applied to several examples of line scans. The left column shows the line scans, and the right column shows the resulting filtered line scans. The first line scan used in this experiment (first row) is an unmodified line scan acquired from a hot steel strip. The next three line scans are a copy of the first line scan corrupted with different noise ratios. The second and third line scans have been corrupted by Gaussian noise with a standard deviation of 3 and 6, respectively. The fourth line scan has been corrupted with 2% random-valued impulse noise.

Fig. 15

Line scans filtered using the proposed approach based on spatial overlapping.

It can be seen that the proposed filtering approach based on spatial overlapping reduces the noise while preserving the integrity of edges and the detail of the information in the line scan. The spatial overlapping filtering does not produce a finely smoothed output; however, it is able to reduce noise effectively. It is important to take into account that the most common cause of noise in this kind of thermographic images are emissivity variations, and this kind of noise cannot be detected from a single line scan. Therefore, the lack of smoothing in the first line scan is considered a success, because a filtering of these slight variations, which could have been caused from real temperature differences, could mean losing important information.

The preservation of edges and detailed information has a drawback: the filter is not able to eliminate high-magnitude Gaussian noise or impulsive noise completely. However, this is a minor issue, first, because this is not a very common noise in thermographic line scans and, second, because the results of the intrascan filtering will be processed by the interscan filtering, which can take care of any noise not eliminated.

Figure 16 shows the results of the average filter and the median filter applied to the line scans shown in the first column of Fig. 15. The window size of both filters was chosen to be 40, 5% of the number of samples in the line scan. The average filter produces a smoothed output, eliminating all the noise in the line scans. However, the edges are not preserved and the slight variations in the center of the line scans are also removed. The median filter also reduces noise, but the output lacks the detailed information of the spatial overlapping filter. Also, the output of the median filter seems aliased, losing the continuity of the signal in some parts.

Fig. 16

Line scans filtered using the average and the median filter.

Figure 17 shows a comparison between the original line scan and the filtered output produced by the spatial overlapping, the average and the median filter. Figure 18 shows the same information but focused on the right edge of the line scan to see the differences between them more clearly. The detailed graph clearly shows how the average filter misses the edge, providing a smoothed slope that blurs the line scan. The median filter preserves the edge information but fails to preserve information about some of the variations of the line scan. On the other hand, the proposed filter based on spatial overlapping is able to provide accurate output, preserving the information available in the original line scan.

Fig. 17

Comparison of intrascan filters.

Fig. 18

Detail of comparison of intrascan filters (the right edge).

The results show that the proposed filter works properly over line scans with or without noise, preserving the information while reducing noise. However, when the noise is high, the result is not as smooth as the output produced by the average or the median filter. This drawback is minor since the output of the filter will be processed in the next step: the interscan filtering, which can handle noise not removed by this filter.

4.3.

Interscan Filtering Results

Figure 19 shows the results of the proposed interscan filtering approach applied to several examples of thermographic images. The left column shows the original image, and the right shows the resulting filtered image. Noise in the original images is clearly appreciated. Also, it can be seen how the noise is mostly removed in the resulting image. Furthermore, although the noise is removed, the edges are totally preserved.

Fig. 19

Thermographic images filtered using the interscan approach. Left column: original, right column; filtered using interscan.

One of the most common causes of noise in this kind of image is emissivity variation, which distorts temperature measurements. This kind of noise cannot be removed in the intrascan filtering, since the only way to detect it is to take adjacent line scans into account. The proposed interscan filter, however, is able to detect this kind of noise and to remove it. Furthermore, temperature changes that correspond to variations in the manufacturing conditions are preserved. This can be observed in Fig. 19.

Figure 20 shows the results obtained when filtering the images in Fig. 19 using the average (left column) and the median filter (right column). The window size of both filters match the window size used in the previously proposed filter. As can be seen, noise is removed. However, when comparing the output of the proposed interscan filtering with the average and median filters, it can be seen that the noise removal ratio is higher in the proposed approach.

Fig. 20

Thermographic images filtered using the average and the median filter. Left column: filtered using the average; right column: filtered using the median.

A more detailed comparison can be made between Figs. 21 and 22 which show a blowup of a longitudinal section in the center of the first image in Fig. 19. The noise of the original signal is very high, with temperature changes of $20°\mathrm{C}$ from one line scan to the next. Over this signal, the longitudinal section of the filtered images obtained using the proposed approach, the average, and the median are shown. Noise reduction is clearly appreciable in all of the three filtered signals. However, the proposed approach does a better job in the two most important aspects: it removes the noise caused by the emissivity variations better, providing a much more smoothed output, and it preserves the integrity of the edges, providing a sharper output. These two statements are corroborated in Fig. 22, which shows a detail of the performance of the three filters on an edge.

Fig. 21

Comparison of interscan filters.

Fig. 22

Detail of comparison of interscan filters (on an edge).

5.

Real-Time Performance Analysis

The proposed filtering method has been designed to be applied in realtime. The deadline of the filtering task is imposed by the acquisition rate. One line scan must be filtered before a new line scan is acquired. This approach makes it possible to use the filtered output for any purpose, for example, to be used as the feedback for a temperature controller.

In order to analyze the viability of the proposed filtering approach, the time needed by each of the computational tasks involved was measured. The algorithms were programmed in C language and measured running in a Core2 Duo E6600 CPU $\left(2.4\mathrm{GHz}\right)$ with $4\mathrm{MB}$ of cache memory and plenty of RAM. The measuring procedure was as follows:

The results obtained are shown in Table 1 .

Table 1

Time measurements.

A typical infrared line scanner, such as the one used to obtain the images shown in this paper, runs at $100\mathrm{Hz}$ . Therefore, a new line scan is acquired every $10\mathrm{ms}$ , which determines the deadline for the maximum line scan processing time. As can be seen in Table 1, the time necessary to apply the proposed filter is much lower than the maximum time available for the line scan processing. This is mainly due to the design and to the implementation of the filter. First, the intrascan filter only uses information about the ration of intersection between samples and all of that information is precalculated before the filtering operates under real-time conditions. Interscan filtering has been designed to recursively operate the gradient calculation, the adaptive average, and the median in the transitory sections. This reduces the number of operations needed to apply the filter, decreasing the computational cost sharply.

The low computational cost of the algorithms is a great advantage. Enough time is left for other kinds of tasks during the maximum line scan processing time, such as feature extraction, or communications through a network. Furthermore, it makes the execution in low-end embedded computers possible, while meeting the deadline.

6.

Conclusions

In this paper, a real-time noise filtering method for a stream of thermographic line scans has been proposed. The filter is divided in two components: an intrascan filter and an interscan filter. The intrascan filter is based on spatial overlapping using high acquisition rates, and the interscan filter is based on edge detection. Both filters are designed to be applied in realtime, introducing recursive calculations for most of the computationally demanding steps. Results obtained corroborate the success of the proposed filtering approach. Several line scans with different kinds of noise have been filtered using the proposed intrascan filter, and the results have been compared to those obtained with the median and the average. The comparison shows that the performance of the proposed approach is good. A similar approach has been used to test the interscan filtering. Several images have been filtered, and the results have been compared to the average and median filters. In this case, the results show that the proposed interscan approach is able to provide much better results than the other two. First, it removes the noise caused by the emissivity variations better, providing a much more smoothed output. Also, it preserves the integrity of the edges, providing a sharper output.

The real-time performance analysis shows that the algorithms can be executed to provide a filtered output while meeting the deadline of the filtering task, which corroborates the correct real-time design. Also, due to the low computational cost of the filtering method proposed, it could even be implemented in low-end embedded computers.

Real-time noise filtering is very likely to find potential applications in a number of different areas, such as robotics, manufacturing control, or any other application based on the processing of a stream of line scans in realtime. Furthermore, even if the proposed method has been described for thermographic images, its use with images taken from the visible spectrum is straightforward.

The proposed filter is being used in a cold rolling mill to provide the flatness control system with the thermographic profiles with of the manufactured steel. The filtered information received by the flatness control system makes it possible to regulate of the manufacturing conditions that produce irregular temperature patterns.