Multi-object tracking is particularly challenging in many scenarios with similar appearance and frequent occlusions among targets. In this paper, we present an online detection-based multi-object tracking method. In each frame, kernerlized convolution filter are adopted to track isolated and un-occluded targets. To overcoming fixed scale in KCF, trackers are associated with detection responses. If a target is associated with a detection, then the target size is updated by the average of this detection size and the previous estimated size. When occlusions are detected, the multiple interaction among targets is formulated as an optimization problem and we explore two-layer hierarchical Particle Swarm Optimization algorithm for the optimal solution. The first layer is designed for the superficial targets which is visible. The second layer is designed for the bottom occluded targets which can guided by first visible layer and we propose to incorporate the attractive force into the particle evolution process. Experimental results on public datasets demonstrate that our proposed method alleviating drifting problem and effectively reduces ID switches and lost trajectories.