Avoiding potential safety hazard is the primary task of vision-based assistant driving system(ADS). Potential safety
hazard exists in driving individual vehicles. Although these hazards are unexpected, obvious characteristics exist for
vehicles that make them happen, such as: relatively fast speed, changing lanes frequently and being occluded as shuttling
in the busy traffic. All these characteristics go against on-road tracking for the unsafe vehicle. At present, the assistant
driving system is only permitted in the field of obstracle detection and location. However, those systems are not involved
in tracking of vehicles with potential safety hazard. The paper presents an approach to tracking and online learning of
on-road vehicles with potential safety hazard. Further, we improve the method of online learning to the unsafe hazard.
The performance of our tracking algorithm is evaluated on a public benchmark with test data from various challenging
videos on different conditions. The experiment results demonstrate that, in the same condition, our method can obtain
samples more efficiently and lead the classifier to converge more quickly.