Background suppressing is the main technology for infrared target detection. We present a new small infrared target detection (SIRTD) method that is also based on background suppressing. First, a new matrix decomposition model, named harmonic and sparse matrix decomposition (HSMD), is put forward for decomposing an image into a harmonic and a sparse component, which are seen as a background component and a small target component, respectively. Then, an algorithm based on augmented Lagrangian alternating direction method (ALADM) for solving HSMD is described. The main computational cost of the proposed algorithm in each iteration is that of a fast Fourier transform (FFT), which makes the proposed algorithm very fast. By searching for the maximum local energy regions in the target component, the infrared targets can be easily and accurately located. Experimental results on some infrared images show that HSMD solved by ALADM is very suitable for real-time infrared image decomposing and SIRTD.