Infrared images have shortcomings of background noise, few details, and fuzzy edges. Therefore, noise suppression and detail enhancement play crucial roles in the infrared image technology field. To effectively enhance details and eliminate noises, an infrared image processing algorithm based on multiscale feature prior is proposed. First, the maximum a posterior model estimating optimal free-noise results is constructed and discussed. Second, based on the extended 16 high-order differential operators and multiscale features, we propose a structure feature prior that is immune to noises and depicts infrared image features more precisely. Third, with the noise-suppressed image, the final image is enhanced by the improved multiscale unsharp mask algorithm, which enhances details and edges adaptively. Finally, testing infrared images in different signal-to-noise ratio scenes, the effectiveness and robustness of the proposed approach is analyzed. Compared with other well-established methods, the proposed method shows the evident performance in terms of noise suppression and edge enhancement.