Advanced uncooled infrared (IR) sensors with wavelength- or polarization-selectivity are advantageous in applications such as fire detection, gas analysis, hazardous material recognition, biological analysis, and polarimetric imaging. Plasmonic metasurfaces (PM) are potential candidates to realize such functionality over a wide range of wavelength, i.e., middle- to long-wavelength IR regions. In particular, metal-insulator-metal (MIM) PMs are most suitable for image sensor applications because of their small pixel size and simple surface configuration. However, such PMs produce various absorption modes, of which some degrade the wavelength or polarization selectivity. In this study, therefore, control of the absorption modes of PMs was investigated for wavelength- or polarization-selective uncooled IR sensor applications. The PMs produce various absorption modes, namely, localized surface plasmon resonance (LSPR), waveguide, Fabry-Pérot, and plasmonic surface lattice resonance (PSLR) modes. These modes can be controlled by engineering the surface configuration or optimizing the dielectric material. The LSPR mode is appropriate for wide-angle detection because it does not depend on the incident angle, whereas the PSLR mode is a potential candidate for narrowband uncooled IR sensors. The waveguide mode, which can degrade the wavelength and polarization selectivity, can be eliminated using an appropriate dielectric material, such as SiO2 or SiN, for high-performance wavelength- or polarization-selective uncooled IR sensors.