Small surface object detection, an important but challenging task for maritime surveillance applications, usually contends with constant changing weather and lighting conditions in realistic atmospheric scenes. This paper presents a novel clutter-adaptive small surface object detection method for use in harbor or maritime scenarios. First, shared features of small surface objects in various cluttered backgrounds are explored, such as dark channel priors, scene geometry constraints, position priors of potential objects in maritime images, similar variances of background pixels in spatial and temporal directions, local contrast in center-surrounding windows, and so on. Then, using Bayesian inference, a probabilistic model is derived to integrate these shared features into our detection framework. The detection algorithm based on the probabilistic model is finally tested on our dataset, which consists of maritime images taken by an onboard camera under various weather/lighting conditions. Extensive experimental results show that, compared with some latest algorithms, our method is more effective for small surface object detection when the background is unknown or constant changing.