The detection of chemical vapor plumes using passive hyperspectral sensors operating in the longwave infrared is a challenging problem with many applications. For adequate performance, detection algorithms require an estimate of a scene’s background statistics, including the mean and covariance. Diffuse plumes with a large spatial extent are particularly difficult to detect in single-image schemes because of contamination of background statistics by the plume. To mitigate the effects of plume contamination, a first pass of the detector can be used to create a background mask. However, large diffuse plumes are typically not removed by a single pass. Instead, contamination can be reduced by using smoothed detection results as a background mask. In the proposed procedure, a detector bank is run on the cube, and a threshold applied to produce a binary image. The binary image can be modeled as a spatial point process consisting of high density and low density regions. By applying a spatial filter to the detection image, regions with overall higher intensity are detected as containing plume and can be removed from background statistic estimates. The key intuition is that regions with a higher density of hits are more likely to contain plume since plumes are spatially contiguous. We demonstrate with real plume data that this method can drastically improve detection performance over the single-pass method, and explore tradeoffs between different filter sizes and thresholds.