A hardware-software architecture designed to support image processing applications facilitates rapid development of a fully programmable, high-frame-rate image-based tracker by simplifying and codifying system interfaces. Effective image-based trackers require several key system interfaces for optimal performance, including sensor input, line of sight command output, telemetry storage and/or transmission, and mode and parameter control. Trackers implemented in software are increasingly feasible in many applications because of trends in processor technology, and they are essential as investigatory aids for algorithm development because of their ease of modification. In a software- dominant system, however, interfaces often constitute fully 90 percent of the source lines of code -- every line of algorithm implementation ode requires nine support lines to deal with interfaces. Reducing the support overhead by defining simple, consistent interfaces at the level nearest the tracker software allows developers to create new systems more quickly and change external interfaces with less disruption. We present a fully programmable image-based tracker whose major interfaces were built from a library of reusable components, resulting in a significantly reduced development time. It has supported advanced algorithm research on tracking in both TV and high-frame-rate digital video.