Stable tracking point extraction for infrared extended target based on contour smoothing and minimal inscribed circle

Abstract. An approach is proposed to reduce the tracking jitter of the extended target in boost phase for plume tracker in a photoelectric acquisition, tracking, and pointing system. The characteristics of the vehicle imaging are analyzed and the causes of jitters are identified. The target moving direction and its principal axis are combined to calculate the optimal frontal direction. A contour smoothing method based on the chord-arc ratio filtering is introduced to obtain a preliminary extraction point with lower jitters. Then a fine tracking point extraction method based on the minimal inscribed circle of contour after filtering is presented. Experimental results confirm that the proposed method significantly improves the tracking precision and stability.

Stable tracking point extraction for infrared extended target based on contour smoothing and minimal inscribed circle 1 Introduction The plume tracker in a photoelectric acquisition, tracking, and pointing (ATP) system needs to lock a settled tracking point on the target stably for accurate measurement or aiming at the target. 1,2 Many attempts have been made to improve the stability of the tracking point extraction, especially for the extended target. [3][4][5] For targets that extend beyond the track gate, the leading edge tracking algorithm is a robust approach that can quickly find and settle on the target nose. In this way, it guarantees locating the target in the center of the track gate and the background taking up some ratio in the field of view to avoid failure of segmentation 6 or tracking point drift. 7 But due to the airstream disturbance introduced by the tail flame, the segmentation thresholds are influenced easily and instable, especially when the target is a vehicle with strong tail flame. In this situation, the vehicle body is prone to be segmented into multiple fragments even if morphological operations are used. After target feature recognition and association, some small fragments are excluded and the target with the highest degree of confidence is left. Unfortunately, the tracking points extracted based on the leading edge tracking algorithm would be jittery between consecutive frames and introduce severe tracking stability problem. Figure 1 shows this situation.
Besides the basic leading edge tracking algorithm shown in Fig. 2(a), which finds the tip of the target directly, some improved leading edge tracking algorithms have been proposed for reducing the jitter of the tracking point, such as the polynomial fitting algorithm shown in Fig. 2(b), the 19-point leading edge tracking algorithm shown in Fig. 2(c), and the correlation tracking algorithm. 4 However, the result of the polynomial fitting algorithm may not be on the target, and it is sensitive to the segmentation error and time cost. Although the result of the 19-point leading edge tracking algorithm is stable because it calculates the centroid of the region near the tip of the target, if there is tip jitter introduced by big segmentation error, the result is not satisfied yet. Due to the template drift phenomenon, 7 the correlation tracker suffers from gradual drift of the tracking region out of the template and losing the target eventually. Although some algorithms against template drifting have been proposed in recent years, 8,9 they are not able to work well with the target with strong tail flame due to severe airstream disturbance. To reduce jitter and improve tracking precision, it is important to analyze the characteristics of vehicle imaging carefully and find out what introduces the jitter.

Characteristic of Vehicle Infrared Image
In the boost phase of a vehicle, the engine jets high-velocity and high-temperature airstream to acquire enough backward thrust. These airstreams form the tail flame of the vehicle and exhibit the characteristic of extended target in the infrared image. The temperature of the vehicle body increases rapidly as the flight velocity increases and the body imaging becomes more and more distinct. So the whole imaging of the vehicle in boost phase includes both body and plume. From the image acquired, it can be seen that the gray-level distribution of the target in this stage consists of three regions. Figure 3(a) shows the real infrared target image after edge enhancement, and Fig. 3(b) shows the threedimensional gray-level distribution of the original image. The vehicle body is zoomed in to show distinctly in Fig. 3(b), and the gray-level fluctuation along its central axis is shown in Fig. 3(c). From Fig. 3, it can be seen that the first region A is the background with low and uniform gray-level distribution introduced by the atmospheric thermal radiation. The second region is the tail flame, which can be subdivided into three parts further. First, there is a region B1 with the uniform and saturated pixels near the central axis. Along the direction away from the central axis, a boundary layer transition region B2 with distinct contour boundary divides the tail flame into these three parts, and its pixel gray level descends rapidly near the boundary. Outside the region B2, there is a region B3 with lower gray level than B1 but higher grey level than the background region A. The gray-level distribution of the region B3 is uniform on the whole, but there are some disturbances leading to random segmentation error introduced by the instable airstream outside it. The third region C is the vehicle body imaging with gray level between the background and the tail flame. The gray-level distribution in this region is nonuniform and severely fluctuant as shown in Fig. 3(c).

Stable Tracking Point Extraction
The conventional leading edge tracking algorithm finds the target frontal along the target moving direction. 2 It is inaccurate when the target velocity is low relative to the ATP system. In this paper, we define the target frontal by combining the moving direction and the angle of the target principal axis. Defined as in Eq. (1), m i;j is the central moment of the target with i × j order. The ðx;ȳÞ is the target centroid and fðx; yÞ is the gray of image pixel at ðx; yÞ.
ðx −xÞ i ðy −ȳÞ j fðx; yÞ: The moment of inertia of the target and its differential are defined as in Eqs. (2) and (3). m 2;0 , m 0;2 , and m 1;1 are obtained from Eq. (1): ½ðy −ȳÞ cos θ þ ðx −xÞ sin θ 2 fðx; yÞ; (2) The angle of the target principal axis is the angle of the target minimal moment of inertia. So let I 0 ðθÞ ¼ 0, we can get two results θ 1 and θ 2 : From Eqs. (4) and (5), it cannot be decided which one is the angle of the target minimal moment of inertia. So the I 0 0 ðθÞ is introduced to discriminate them: I 0 0 ðθÞ ¼ ½2ðm 2;0 − m 0;2 Þ cos 2θ þ 4m 1;1 sin 2θfðx; yÞ: θ 1 and θ 2 are substituted in Eq. (6), if I 0 0 ðθ i Þ > 0, then this θ i is the angle of the target minimal moment of inertia, which is labeled as θ axis . Then the direction of the target nose could be calculated from Eq. (7). θ v is the target moving direction.
Along the direction of θ n , the tip of the target could be found. Unfortunately, there are severe segmentation errors between consecutive frames, leading to tracking jitter due to the nonuniform gray-level distribution of the target,  especially on the vehicle body. In this paper, a novel method based on the chord-arc ratio filtering for contour smoothing is proposed. After the contour smoothing, the big fluctuations and the spiculate arcs on the target contour are removed. Figure 4(a) shows the sketch of the filtering method. As Fig. 4(a) shows, the contour tracking algorithm is applied first. Then all contour points arranged in order are processed one by one along the same direction, clockwise or anticlockwise. Let us assume that the point processed currently is Pcur and the number of contour points is lenContour. From the next point of Pcur along the processed direction, all points Pi in the circle with radius R and centered at Pcur are searched along the same direction. The value of R is the upper limit length of the chord between the two endpoints of the arc being removed. It decides the max width of the vehicle body that could be eliminated. The length of the chord between Pcur and Pi is calculated and labeled as lenChd½i. lenArc½i is the length of the minor arc between them. Then the chord-arc ratio is calculated as in Eq. (8): ratioCur½i ¼ lenChd½i∕lenArc½i; i ¼ 1: : : N: N is the number of points searched in the circle along the search direction. After completing searching in the circle for the point Pcur, the minimal ratio Ratio min is obtained using Eq. (9) for Pcur. Ratio min ¼ minfratioCur½i; i ¼ 1: : : Ng: If Ratio min < Ratio th and corresponding lenChd½i meets lenChd½i > lenChd th , then the point Pcur is labeled as should be connected, and the value of the Ratio min , the points Pcur and Pi are recorded. The Ratio th and lenChd th are thresholds for the chord-arc ratio and the length of the chord, respectively. The parameter lenChd th is introduced to obtain smoother results after filtering and reduce remaining irregular spiculate arc. It decides the minimal width of the vehicle body that could be eliminated.
After completing the minimal chord-arc ratio calculation for every contour point, the second traversal is applied again. If the point Pcur; processed currently is recorded as should be connected, then the contour points on the minor arc between it and the point Pi recorded are removed, and a new straight line connecting them is inserted in to form a new arc. After the second traversal is completed, a smoother contour is obtained. The contour smoothing result of a real target contour is shown in Fig. 4(b). The yellow pixels are contour points reserved after filtering, and the white pixels are contour points that have been eliminated. It can be seen that the unstable and spiculate part of the contour that is introduced by the vehicle body has been removed and the portion reserved is the smoother contour of the plume.
As Fig. 5(a) shows, although most of the big fluctuations on contour have been removed after smoothing, there are still some small fluctuations remaining as Ratio th and lenChd th introduced are still sensitive to the random segmentation error between consecutive frames. Then the tracking point  In the paper, Rb is 3 × Width f max ∕4, where Width f max is the max width of the tail flame. Obviously, the point Pb is more stable than Pa because it is an ensemble average of the frontal part of the target. But based on the analysis of the plume imaging characteristic, it can be seen that because most of the plume pixels are saturated inside the boundary layer transition region B2, the contribution of the centroid of the frontal region cannot compensate for influence of the tip (Pa) jitter, because the tip (Pa) jitter introduces regions used for calculating centroid shifting in consecutive frames. Based on the fact that the region B3 has more uniform and higher gray-level distribution than the pixels outside the boundary between B3 and background region A, a tracking point extraction method based on the minimal inscribed circle of the frontal part of the plume contour is presented for reducing jitter further. After the minimal inscribed circle Cins centered at Pb of the contour is obtained, the tracking point Pt is extracted as the cross shown in Fig. 5(a), which is the intersection of Cins and Lp near the tip. Lp is the straight line with the angle of the target principal axis and through the point Pb. From Figs. 5(a) and 5(b), it can be seen that there is even severe segmentation error on the vehicle body between consecutive frames; however, when Pa and Pb jitters, only the radius of Cins changes and the tracking point Pt is stable. Figure 6 shows the tracking point extraction results and the jitter calculated. The revised 19-point leading edge algorithm adopts the point Pa as the tracking point. The tracking jitter is the difference between the original synthetical value and the result from the least squares fitting of the tracking point. Figures 6(a) and 6(b) show the results of several consecutive  The algorithm proposed was implemented on the DSP TMS320C6455 (1.2 GHZ) for real-time tracking point extraction as one module of the image processing and target tracking machine in the ATP system. For 320 × 256 pixel 14-bit real infrared image, the target features and the time cost are shown in Table 1.

Conclusion
In conclusion, a novel method based on the contour smoothing and minimal inscribed circle is presented for improving the stability of the tracking point extraction. It is insensitive to the segmentation threshold and error. The theoretical analysis and the experimental results show that the tracking point jitter is reduced and the tracking stability is improved dramatically based on the method proposed.
Besides the situation of tracking the plume target talked about in the article, the proposed method could be used for tracking the infrared rigid target like a plane too. The contour smoothing method based on the chord-arc ratio filtering could be extensively used in other image analysis scenes such as the object defect detection, the object recognition, and so on.  Qinzhang Wu is a professor and PhD supervisor of the IOE, Chinese Academy of Sciences. His research interests include high-resolution imaging, image understanding, multitarget tracking, data fusion, and the integration technology for photoelectric imaging and tracking system.