Short-term forecasting of cloud distribution within a sequence of all-sky images is an important issue in
meteorological area. In this work, a cloud image forecasting system is designed, which includes three steps---cloud
detection, cloud matching and motion estimation. We treat cloud detection as a classification problem based on Linear
Discriminant Analysis. During the matching, a set of Speed Up Robust Features (SURF) are extracted to represent the
cloud, then clouds are matched by computing correspondences between SURF features. Finally, affine transform is
applied to estimate the motion of cloud. This local features based method is capable of predicting the rotation and scaling
of cloud, while the traditional method is only limited to translational motion. Objective evaluation results show higher
accuracy of the proposed method compared with some other algorithms.