This paper develops an algorithm for autonomous tracking of a person (target) within a crowded and temporally dynamic scene using a multispectral imaging system. The camera is stationary, the field of view is static, and the sensor pixel footprint is on the order of one inch. The operator designates the target to be tracked by selecting a single target-pixel in the first image frame, preferably close to the center of mass of the observable portion of the target in that particular frame. Following the initial designation, the algorithm provides tracking of the target in real-time autonomously with minimal latency. The tracking algorithm is based on a novel temporally adaptive spatial-spectral filter bank used to detect target presence or lack thereof in the field-of-regard of the video frame produced by the multispectral camera. The theory of the temporally adaptive spatial-spectral filter is based on an extension of our earlier work on the enhanced matched filter bank (EMFB). The concept of EMFB is founded on the theory of spatial matched filters, which is the optimal correlation filter for detection of a known image corrupted by noise.