In the intelligence community, aerial video has become one of the fastest growing data sources and it has been
extensively used in intelligence, surveillance, reconnaissance, tactical and security applications. This paper
presents a tracking approach to detect moving vehicles and person in such videos taken from aerial platform.
In our approach, we combine the layer segmentation approach with background stabilization and post-tracking
refinement to reliably detect small moving objects at the relatively low processing speed. For each individual
moving object, a corresponding layer is created to maintain an independent appearance and motion model
during the tracking process. After the online tracking process, we apply a post-tracking refinement process to
link the track fragments into a long consistent track ID to further reduce false alarm and increase detection rate.
Furthermore, a vehicle and person classifier is also integrated into the approach to identify the moving object
categories. The classifier is based on image histogram of gradient (HOG), which is more reliable to illumination
variation or camera automatic gain change. Finally, we report the results of our algorithms on a large scale of
EO and IR data set collected from VIVID program, and the results show that our approach achieved a good
and stable tracking performance on the data set that is more than eight hours.