A novel method to detect flames in infrared (IR) video is proposed. Image regions containing flames appear as bright regions in IR video. In addition to ordinary motion and brightness clues, the flame flicker process is also detected by using a hidden Markov model (HMM) describing the temporal behavior. IR image frames are also analyzed spatially. Boundaries of flames are represented in wavelet domain and the high frequency nature of the boundaries of fire regions is also used as a clue to model the flame flicker. All of the temporal and spatial clues extracted from the IR video are combined to reach a final decision. False alarms due to ordinary bright moving objects are greatly reduced because of the HMM-based flicker modeling and wavelet domain boundary modeling.