5 November 2013 Stable tracking point extraction for infrared extended target based on contour smoothing and minimal inscribed circle
Author Affiliations +
Optical Engineering, 52(11), 113101 (2013). doi:10.1117/1.OE.52.11.113101
An approach is proposed to reduce the tracking jitter of the extended target in boost phase for plume tracker in a photoelectric acquisition, tracking, and pointing system. The characteristics of the vehicle imaging are analyzed and the causes of jitters are identified. The target moving direction and its principal axis are combined to calculate the optimal frontal direction. A contour smoothing method based on the chord-arc ratio filtering is introduced to obtain a preliminary extraction point with lower jitters. Then a fine tracking point extraction method based on the minimal inscribed circle of contour after filtering is presented. Experimental results confirm that the proposed method significantly improves the tracking precision and stability.
Lei, Yang, Jiang, and Wu: Stable tracking point extraction for infrared extended target based on contour smoothing and minimal inscribed circle



The plume tracker in a photoelectric acquisition, tracking, and pointing (ATP) system needs to lock a settled tracking point on the target stably for accurate measurement or aiming at the target.1,2 Many attempts have been made to improve the stability of the tracking point extraction, especially for the extended target.34.5 For targets that extend beyond the track gate, the leading edge tracking algorithm is a robust approach that can quickly find and settle on the target nose. In this way, it guarantees locating the target in the center of the track gate and the background taking up some ratio in the field of view to avoid failure of segmentation6 or tracking point drift.7 But due to the airstream disturbance introduced by the tail flame, the segmentation thresholds are influenced easily and instable, especially when the target is a vehicle with strong tail flame. In this situation, the vehicle body is prone to be segmented into multiple fragments even if morphological operations are used. After target feature recognition and association, some small fragments are excluded and the target with the highest degree of confidence is left. Unfortunately, the tracking points extracted based on the leading edge tracking algorithm would be jittery between consecutive frames and introduce severe tracking stability problem. Figure 1 shows this situation.

Fig. 1

The tracking point jitter between consecutive frames. (a) Segmentation result of the 1231th frame. (b) Extraction result in the 1231th frame. (c) Segmentation result of the 1232th frame. (d) Extraction result in the 1232th frame.


Besides the basic leading edge tracking algorithm shown in Fig. 2(a), which finds the tip of the target directly, some improved leading edge tracking algorithms have been proposed for reducing the jitter of the tracking point, such as the polynomial fitting algorithm shown in Fig. 2(b), the 19-point leading edge tracking algorithm shown in Fig. 2(c), and the correlation tracking algorithm.4 However, the result of the polynomial fitting algorithm may not be on the target, and it is sensitive to the segmentation error and time cost. Although the result of the 19-point leading edge tracking algorithm is stable because it calculates the centroid of the region near the tip of the target, if there is tip jitter introduced by big segmentation error, the result is not satisfied yet. Due to the template drift phenomenon,7 the correlation tracker suffers from gradual drift of the tracking region out of the template and losing the target eventually. Although some algorithms against template drifting have been proposed in recent years,8,9 they are not able to work well with the target with strong tail flame due to severe airstream disturbance. To reduce jitter and improve tracking precision, it is important to analyze the characteristics of vehicle imaging carefully and find out what introduces the jitter.

Fig. 2

The tracking point extraction algorithm for extended target with tail flame. (a) Basic leading tracking algorithm. (b) Polynomial fitting algorithm. (c) 19-point leading tracking algorithm.



Characteristic of Vehicle Infrared Image

In the boost phase of a vehicle, the engine jets high-velocity and high-temperature airstream to acquire enough backward thrust. These airstreams form the tail flame of the vehicle and exhibit the characteristic of extended target in the infrared image. The temperature of the vehicle body increases rapidly as the flight velocity increases and the body imaging becomes more and more distinct. So the whole imaging of the vehicle in boost phase includes both body and plume. From the image acquired, it can be seen that the gray-level distribution of the target in this stage consists of three regions. Figure 3(a) shows the real infrared target image after edge enhancement, and Fig. 3(b) shows the three-dimensional gray-level distribution of the original image. The vehicle body is zoomed in to show distinctly in Fig. 3(b), and the gray-level fluctuation along its central axis is shown in Fig. 3(c).

Fig. 3

The gray-level distribution of the vehicle. (a) Target image after edge enhancement and different regions. (b) Three-dimensional distribution of the original image and the vehicle body is zoomed in. (c) Gray-level fluctuation along the central axis of the vehicle body.


From Fig. 3, it can be seen that the first region A is the background with low and uniform gray-level distribution introduced by the atmospheric thermal radiation. The second region is the tail flame, which can be subdivided into three parts further. First, there is a region B1 with the uniform and saturated pixels near the central axis. Along the direction away from the central axis, a boundary layer transition region B2 with distinct contour boundary divides the tail flame into these three parts, and its pixel gray level descends rapidly near the boundary. Outside the region B2, there is a region B3 with lower gray level than B1 but higher grey level than the background region A. The gray-level distribution of the region B3 is uniform on the whole, but there are some disturbances leading to random segmentation error introduced by the instable airstream outside it. The third region C is the vehicle body imaging with gray level between the background and the tail flame. The gray-level distribution in this region is nonuniform and severely fluctuant as shown in Fig. 3(c).


Stable Tracking Point Extraction

The conventional leading edge tracking algorithm finds the target frontal along the target moving direction.2 It is inaccurate when the target velocity is low relative to the ATP system. In this paper, we define the target frontal by combining the moving direction and the angle of the target principal axis. Defined as in Eq. (1), mi,j is the central moment of the target with i×j order. The (x¯,y¯) is the target centroid and f(x,y) is the gray of image pixel at (x,y).



The moment of inertia of the target and its differential are defined as in Eqs. (2) and (3). m2,0, m0,2, and m1,1 are obtained from Eq. (1):





The angle of the target principal axis is the angle of the target minimal moment of inertia. So let I(θ)=0, we can get two results θ1 and θ2:





From Eqs. (4) and (5), it cannot be decided which one is the angle of the target minimal moment of inertia. So the I(θ) is introduced to discriminate them:



θ1 and θ2 are substituted in Eq. (6), if I(θi)>0, then this θi is the angle of the target minimal moment of inertia, which is labeled as θaxis. Then the direction of the target nose could be calculated from Eq. (7). θv is the target moving direction.



Along the direction of θn, the tip of the target could be found. Unfortunately, there are severe segmentation errors between consecutive frames, leading to tracking jitter due to the nonuniform gray-level distribution of the target, especially on the vehicle body. In this paper, a novel method based on the chord-arc ratio filtering for contour smoothing is proposed. After the contour smoothing, the big fluctuations and the spiculate arcs on the target contour are removed. Figure 4(a) shows the sketch of the filtering method.

Fig. 4

The contour smoothing method and result based on chord-arc ratio filtering. (a) The sketch of the contour smoothing. (b) The result of contour smoothing.


As Fig. 4(a) shows, the contour tracking algorithm is applied first. Then all contour points arranged in order are processed one by one along the same direction, clockwise or anticlockwise. Let us assume that the point processed currently is Pcur and the number of contour points is lenContour. From the next point of Pcur along the processed direction, all points Pi in the circle with radius R and centered at Pcur are searched along the same direction. The value of R is the upper limit length of the chord between the two endpoints of the arc being removed. It decides the max width of the vehicle body that could be eliminated. The length of the chord between Pcur and Pi is calculated and labeled as lenChd[i]. lenArc[i] is the length of the minor arc between them. Then the chord-arc ratio is calculated as in Eq. (8):



N is the number of points searched in the circle along the search direction. After completing searching in the circle for the point Pcur, the minimal ratio Ratiomin is obtained using Eq. (9) for Pcur.



If Ratiomin<Ratioth and corresponding lenChd[i] meets lenChd[i]>lenChdth, then the point Pcur is labeled as should be connected, and the value of the Ratiomin, the points Pcur and Pi are recorded. The Ratioth and lenChdth are thresholds for the chord-arc ratio and the length of the chord, respectively. The parameter lenChdth is introduced to obtain smoother results after filtering and reduce remaining irregular spiculate arc. It decides the minimal width of the vehicle body that could be eliminated.

After completing the minimal chord-arc ratio calculation for every contour point, the second traversal is applied again. If the point Pcur; processed currently is recorded as should be connected, then the contour points on the minor arc between it and the point Pi recorded are removed, and a new straight line connecting them is inserted in to form a new arc. After the second traversal is completed, a smoother contour is obtained. The contour smoothing result of a real target contour is shown in Fig. 4(b). The yellow pixels are contour points reserved after filtering, and the white pixels are contour points that have been eliminated. It can be seen that the unstable and spiculate part of the contour that is introduced by the vehicle body has been removed and the portion reserved is the smoother contour of the plume.

As Fig. 5(a) shows, although most of the big fluctuations on contour have been removed after smoothing, there are still some small fluctuations remaining as Ratioth and lenChdth introduced are still sensitive to the random segmentation error between consecutive frames. Then the tracking point extracted based on the basic leading edge tracking algorithm, the polynomial fitting algorithm, or the 19-point leading edge tracking algorithm is still jittery. For extracting a stable tracking point further, a novel method based on the minimal inscribed circle of contour after filtering is proposed.

Fig. 5

The tracking point extraction results of consecutive frames based on the minimal inscribed circle. (a) The N’th frame result. (b) The (N+1)’th frame result.


First, a preliminary frontal point Pa is obtained with the 19-point leading edge tracking algorithm. Because it calculates the tracking point based on the result after contour smoothing, it is called revised 19-point leading edge tracking. Due to the effect of the contour smoothing, the jitter of Pa is lower than the result without smoothing. Then the centroid point Pb of the circle region with radius Rb and centered at the point Pa is calculated. The value of Rb is chosen based on the max width of the flame. A suitable value of Rb could improve the stability of Pb calculated. If Rb is too small, the stability of Pb is influenced easily by the residual fluctuation on the vehicle body contour after filtering. If it is too big, it would be influenced by the instable airstream on the caudal region of the tail flame. In the paper, Rb is 3×Widthfmax/4, where Widthfmax is the max width of the tail flame. Obviously, the point Pb is more stable than Pa because it is an ensemble average of the frontal part of the target. But based on the analysis of the plume imaging characteristic, it can be seen that because most of the plume pixels are saturated inside the boundary layer transition region B2, the contribution of the centroid of the frontal region cannot compensate for influence of the tip (Pa) jitter, because the tip (Pa) jitter introduces regions used for calculating centroid shifting in consecutive frames. Based on the fact that the region B3 has more uniform and higher gray-level distribution than the pixels outside the boundary between B3 and background region A, a tracking point extraction method based on the minimal inscribed circle of the frontal part of the plume contour is presented for reducing jitter further. After the minimal inscribed circle Cins centered at Pb of the contour is obtained, the tracking point Pt is extracted as the cross shown in Fig. 5(a), which is the intersection of Cins and Lp near the tip. Lp is the straight line with the angle of the target principal axis and through the point Pb. From Figs. 5(a) and 5(b), it can be seen that there is even severe segmentation error on the vehicle body between consecutive frames; however, when Pa and Pb jitters, only the radius of Cins changes and the tracking point Pt is stable.


Experimental Results

Figure 6 shows the tracking point extraction results and the jitter calculated. The revised 19-point leading edge algorithm adopts the point Pa as the tracking point. The tracking jitter is the difference between the original synthetical value and the result from the least squares fitting of the tracking point. Figures 6(a) and 6(b) show the results of several consecutive frames on x and y, respectively, based on the revised 19-point leading edge tracking algorithm Pa, the center of the minimal inscribed circle Pb, and the ultimate tracking point Pt extracted by the method proposed in this paper. Figures 6(c) and 6(d) show the jitter comparison among the algorithms in the extraction results of several consecutive frames on the azimuth and the elevation, respectively. In the test infrared image sequence, one pixel occupies 15.23 arc sec. Based on the proposed method, the results show the jitters are no more than 0.077 pixel RMS (1.17 in. RMS) on azimuth and 0.25 pixel RMS (3.87 in. RMS) on elevation and the stability is improved 15.3 times and 21.4 times, respectively. Obviously, it achieves better performance in tracking stability.

Fig. 6

The tracking point extraction and jitter comparison. (a) The pixel extraction results on x. (b) The pixel extraction results on y. (c) The jitter on azimuth. (d) The jitter on elevation.


The algorithm proposed was implemented on the DSP TMS320C6455 (1.2 GHZ) for real-time tracking point extraction as one module of the image processing and target tracking machine in the ATP system. For 320×256pixel 14-bit real infrared image, the target features and the time cost are shown in Table 1.

Table 1

Target features and the time cost.

FrameWidthfmaxNum of contour pointsTarget areaTime cost (ms)



In conclusion, a novel method based on the contour smoothing and minimal inscribed circle is presented for improving the stability of the tracking point extraction. It is insensitive to the segmentation threshold and error. The theoretical analysis and the experimental results show that the tracking point jitter is reduced and the tracking stability is improved dramatically based on the method proposed.

Besides the situation of tracking the plume target talked about in the article, the proposed method could be used for tracking the infrared rigid target like a plane too. The contour smoothing method based on the chord-arc ratio filtering could be extensively used in other image analysis scenes such as the object defect detection, the object recognition, and so on.


The authors gratefully acknowledge support by the 863 Project from Science and Technology Department (No. G107309, No. G107302). The authors also want to express gratitude to the anonymous reviewers whose thoughtful comments and suggestions improved the quality of the article.



B. L. Ulich, “Overview of acquisition, tracking, and pointing system technologies,” Proc. SPIE 887, 40–63 (1988).PSISDG0277-786Xhttp://dx.doi.org/10.1117/12.944208Google Scholar


J. N. Sanders-Reed, “Multi-target, multi-sensor, closed loop tracking,” Proc. SPIE 5430, 1–19 (2004).PSISDG0277-786Xhttp://dx.doi.org/10.1117/12.518557Google Scholar


P. D. Hill, “Real-time video edge tracking algorithms,” Proc. SPIE 1950, 141–151 (1993).PSISDG0277-786Xhttp://dx.doi.org/10.1117/12.156599Google Scholar


J. W. BukleyR. M. Cramblitt, “Comparison of image processing algorithms for tracking illuminated targets,” Proc. SPIE 3692, 234–243 (1999).PSISDG0277-786Xhttp://dx.doi.org/10.1117/12.352866Google Scholar


Z. PengQ. ZhangA. Guan, “Extended target tracking using projection curves and matching pel count,” Opt. Eng. 46(6), 064401 (2007).OPEGAR0091-3286http://dx.doi.org/10.1117/1.2746913Google Scholar


F. GallandP. Réfrégier, “Information-theory-based snake adapted to multi region objects with different noise models,” Opt. Lett. 29(14), 1611–1613 (2004).OPLEDP0146-9592http://dx.doi.org/10.1364/OL.29.001611Google Scholar


J. Ahmedet al., “Real-time edge-enhanced dynamic correlation and predictive open-loop car-following control for robust tracking,” Mach. Vis. Appl. 19(1), 1–25 (2008).MVAPEO0932-8092http://dx.doi.org/10.1007/s00138-007-0072-4Google Scholar


J. Y. PanB. Hu, “Robust object tracking against template drift,” in Proc. IEEE Int. Conf. on Image Processing, Vol. 3, pp. III-353–III-356, IEEE, San Antonio, TX (2007).Google Scholar


T. HanM. LiuT. Huang, “A drifting-proof framework for tracking and online appearance learning,” in Proc. IEEE Workshop Applications of Computer Vision, pp. 1–10, IEEE, Austin, TX (2007).Google Scholar



Tao Lei received his MS degree from the Graduate School of Chinese Academy of Science in 2006 and PhD degree in signal and information processing from the same school in 2013. From 2006, he worked at the Institute of Optics and Electronics (IOE), Chinese Academy of Sciences. Now he is an associate researcher. His research interests include signal processing, image processing, target recognition and tracking, and real-time data processing.


Sihan Yang received her MS degree in computer science from Chengdu University of Technology in 2006. She has been a PhD candidate since 2008 and is also a lecturer in the same university. Her current research interests include image processing, computer vision, and target recognition and tracking.


Ping Jiang received his MS degree from the University of Electronic Science and Technology of China in 2004 and his PhD degree from the Sichuan University in 2011. From 2002, he worked at the IOE, Chinese Academy of Sciences. Now he is an associate researcher. His current research interests are in the areas of target tracking, machine vision, and information fusion.


Qinzhang Wu is a professor and PhD supervisor of the IOE, Chinese Academy of Sciences. His research interests include high-resolution imaging, image understanding, multitarget tracking, data fusion, and the integration technology for photoelectric imaging and tracking system.

Tao Lei, Sihan Yang, Ping Jiang, Qinzhang Wu, "Stable tracking point extraction for infrared extended target based on contour smoothing and minimal inscribed circle," Optical Engineering 52(11), 113101 (5 November 2013). http://dx.doi.org/10.1117/1.OE.52.11.113101
Submission: Received ; Accepted

Detection and tracking algorithms

Infrared radiation



Target acquisition

Infrared imaging

Optical tracking

Back to Top