Heterogeneous camera based surveillance systems provide us with a more robust tracking of objects. To take advantage
of additional cameras, it is necessary to establish geometrical relationship between the cameras and relationship between
an object and a camera. This paper presents an algorithm that can track a non-rigid objects in real-time in the night watch
system which does not contain sufficient light. The proposed method adopted hierarchical active shape model(ASM) for
real-time tracking and adaptive landmark point assignment for reducing computational load at each level. Active Shape
Model is robust for tracking non-rigid objects and overcomes the occlusion, because it changes an average shape of an
object with trained contour information of an object. This proposed tracking algorithm uses the information from CCD
sensor for tracking objects in the day time, and uses the information form IR sensor for tracking objects in the night time.
When the perfect occlusion occurs, the proposed algorithm predicts movements of an object using the historical tracking
information and it can keep the object tracking. Through the results of this experiment, we found out that we can track an
object both day and night with an trained contour information of an object, and confirm that robust tracking can be done
in a part occlusion. Therefore, the proposed algorithm we will develop a real-time region alignment algorithm for a
heterogeneous camera-based surveillance system under a complex environment.