Aiming at the requirement of high frame frequency, real-time processing of detecting and tracking target in modern IR defense and detecting system, this article introduces the design and engineering implementation of a portable IR target automatic detecting and tracking system. Firstly analyzes the overall structure and introduce the composition and function in detail. Secondly puts forward the flow of low-altitude infrared small target detecting and tracking algorithm, then researches the background suppressing and detecting technology of far-distance weak small targets with low SCR. Finally aiming at the key technology of real-time processing with the limit of volume and power, a project of real-time signal processor that is reconfigurable based on DSP+FPGA is proposed and implemented, which sends the images and detecting results to host computer by USB2.0 interface. Besides, the hardware architecture, the target detecting flow in DSP, the driver and the firmware of USB2.0 are introduced too. The experimental result shows that this system can detect and track the target automatically and accurately, which satisfy the requirement of the real-time performance of the system.