Human factors issues related to head mounted imaging systems have driven the requirements for system latency to nearly the bounds of sensor physics. Image processing must therefore be performed in an envelope that is ever decreasing in size. This paper presents a complete method for intelligent fusion of a long wave infrared and visible sensor, including contrast enhancement in both spectrums, with end to end processing latency of less than 1 millisecond. The use of image statistics and opponent color theory allows fusion with minimal computational resources and latency. This algorithm has demonstrated performance without inducing any noticeable human factors issues during user trials.