15 June 2012 Photometric limits for digital camera systems
Author Affiliations +
J. of Electronic Imaging, 21(2), 020501 (2012). doi:10.1117/1.JEI.21.2.020501
Abstract
Image sensors for digital cameras are built with ever decreasing pixel sizes. The size of the pixels seems to be limited by technology only. However, there is also a hard theoretical limit for classical video camera systems: During a certain exposure time only a certain number of photons will reach the sensor. The resulting shot noise thus limits the signal-to-noise ratio. In this letter we show that current sensors are already surprisingly close to this limit.
Schöberl, Brückner, Foessel, and Kaup: Photometric limits for digital camera systems

1.

Introduction

The steady progress in semiconductor technology allows the manufacturing of smaller and smaller structures and image sensors with ever shrinking pixel sizes. One can get the impression that the pixel size is just limited by the technology and even smaller pixels are desirable. Today, consumer products with pixel sizes dp=1.4μm are already on the market and devices with dp=1.1μm are in production.1 In comparison, photo receptors in the human eye are reported to be larger than 3 μm.2

The general modeling of light is well understood,3 and simulation with commercial tools like ISET4 is possible. In contrast, this letter addresses parameters like aperture and pixel size and their photometric consequences for modeling the amount of light that is available in a digital video camera system. One of the design parameters is the resulting image quality. With small pixels only a few photons will hit a single pixel during an exposure period and the signal-to-noise power ratio (SNR) will be poor due to shot noise.5 Apart from all technological limitations, this physical boundary limits the performance of today’s video cameras.

2.

Image Acquisition Model

The scene radiates a certain amount of light. This is described by an average radiance in object space Lobj. The sensor sees an effective amount of light equivalent to the cone with a solid angle Ω as shown in Fig. 1(a). This cone is defined by the sphere of radius equal to the focal length f and a circular aperture disk with diameter D. The solid angle thus calculates to6

(1)

Ω=π·(D/2)2f2=π4·(f/D)2[sr].
The sensor receives an irradiance I

(2)

I=ηlens·Ω·Lobj[W·m2],
for a lens with optical transmittance ηlens.

Fig. 1

Parameters of (a) focal length f, aperture diameter D and resulting solid angle Ω and (b) quadratic pixels with fill factor γff.

JEI_21_2_020501_f001.png

On the sensor some area is used for interconnects and transistors so that only some of the area is sensitive to light. Figure 1(b) shows pixels of size dp. The ratio of active to total areas is expressed as an effective sensor fill factor γff. Even with clever manufacturing like micro-lenses or back side illumination, γff<1 holds. A single pixel thus captures a certain amount of radiant power (radiant flux) Φpix of the sensor irradiance

(3)

Φpix=dp2·γff·I[W].
A single photon of wavelength λ has the energy hcλ with the speed of light c and Planck’s constant h. The radiant flux Φ thus consists of Nphot photons

(4)

Nphot=1hcλ·τexp·Φpix,
during a certain time interval (exposure time) of τexp. In the photoreceptor only some of these photons are converted into electrons Nelec=ηqe·Nphot while others are not, due to reflection, recombination and other material interactions. The conversion rate is expressed as quantum efficiency ηqe.7 The electrons are then collected in the pixel. Although we will see Nelec, on average, the charge is still quantized and the actual number of electrons is subject to shot noise due to the occurrence of random events. For N electrons the associated shot noise is of strength N.5 As NelecΦpix, signal power is represented with Nelec and SNR thus calculates to

(5)

SNR=Nelec/Nelec=Nelec.
In CCD or CMOS technology there are further sources of sensor noise,8 which are neglected in the ideal case.

SNR is a parameter that is directly visible in the final images. For answering the original question, we can combine the above equations. This leads to

(6)

dp,min=SNR2·hcληqe·γff·τexp·4·(f/D)2ηlens·π·Lobj.

3.

Results for Ideal System

At first we assume ideal technology. A typical indoor scene is illuminated with a luminance of Lv=100cdm2.9 For the peak sensitivity of the human eye at a wavelength of λ=555nm the SI unit candela is defined10 as radiant intensity of 1/683Wsr1. The radiance in object space is then

(7)

Lobj=100·16830.146Wsr1m2.
We further assume a perfectly transparent lens with ηlens=1, a wide aperture f/D=2.8, fill factor γff=1 and quantum efficiency ηqe=1. For achieving typical video frame rates a maximum exposure time of τexp=0.03s is used. For a human observer, images without visible noise are preferred. From psychophysical studies a thousand-photon limit is reported as the threshold for visibility of shot noise.5 We therefore set SNR=100032. With green light with λ=555nm the minimum pixel size calculates to dp,min=0.9μm.

The influence of different apertures is shown in Fig. 2. With larger aperture diameters, even smaller pixels can be used. A variation of luminance is also possible: In practice, the human color perception (photoptic vision) starts at Lv=3cdm2.9 The luminance in daylight exterior scenarios is typically Lv=104cdm2.9 The resulting minimum pixel sizes thus range from 5 to 0.09 μm as shown in Fig. 3.

Fig. 2

Minimum pixel sizes for photon limited system with varying apertures, ideal system with SNR=1000 and scene with luminance of Lv=100cdm2, dashed line for f/D=2.8.

JEI_21_2_020501_f002.png

Fig. 3

Minimum pixel sizes for photon limited system with varying luminance, ideal system with SNR=1000 and aperture f/D=2.8, dashed line for Lv=100cdm2.

JEI_21_2_020501_f003.png

4.

Radiometric Modeling

Up to now, we used monochromatic light only. We now extend this and also include the spectral distribution of light. Again, we start with a scene with a luminance of Lv=100cdm2. Now, the light is made up of radiation from a light bulb. This is modeled as a black body at a certain color temperature T and a spectral radiance of

(8)

Lobj,λ(λ,T)=L0·2hc2λ5·1ehcλkT1[W·sr1·m3].
With the photoptic luminous efficiency function11 Vm, we set

(9)

Km·0Vm·(λ)·Lobj,λ(λ,T)dλ=!Lv,
with Km=683lmW1. The resulting normalized spectral radiance Lobj,λ(λ,T) is now perceived by the human eye as a luminance of Lv=100cdm2. Figure 4 shows the resulting set of normalized spectral radiances for typical color temperatures.

Fig. 4

Spectral radiance of black bodies with temperatures T, intensity scaled to be perceived as luminance of 100cdm2.

JEI_21_2_020501_f004.png

Today, most cameras are used to capture scenes for later viewing by a human. The camera should therefore create a representation of the scene that is similar to that of the human visual system. We simulate an ideal camera with the spectral sensitivity curves based on the Stockman and Sharpe cone measurements of the human eye.12 The corresponding spectral sensitivity functions for long (L), medium (M) and short (S) wavelengths are shown in Fig. 5. However, we assume an ideal camera with ideal color filters and material without any attenuation (ηqe=1) at peak efficiency.

Fig. 5

Sensitivity functions of 10-deg cone fundamentals for L, M and S cone and luminous efficiency function Vm.

JEI_21_2_020501_f005.png

In Table 1, the resulting minimum pixel sizes are shown for the radiometric simulation. The luminosity case with monochromatic light at λ=555nm corresponds to the ideal simulation from above. There is less than 10% error for the simulation with L and M cones compared to the luminosity. This is plausible from the high similarity of the respective sensitivity curves. However, the capturing of blue light (short wavelengths with cone S) requires larger pixels. At short wavelengths, the individual photons have a higher energy and thus, there are fewer for a given radiant flux. This explains the problem of inferior performance of blue color channels in typical digital cameras. The extreme case of observing monochromatic green light with a short wavelength sensitivity leads to even fewer photons and would require pixels with 26 μm. In general, the monochromatic calculation is only slightly optimistic but gives a good approximation to a radiometric computation.

Table 1

Minimum pixel sizes (in μm) based on radiometric calculations for light sources with black body radiation of temperature T and monochromatic light source.

Light sourceCone LCone MCone SLuminosity
T=3200  K0.861.042.130.89
T=4500  K0.891.021.640.90
T=5600  K0.901.011.450.90
T=6400  K0.911.011.360.90
λ=555  nm0.920.9226.000.90

5.

Results with Current Technology

The above numbers represent the theoretical limit for ideal sensors. In practice, a real world camera does not achieve these numbers. For example, a highly optimized three layer stacked image sensor is reported by Hannebauer et al.13 For pixels of size dp=4.8μm a high fill factor of γff=0.95 and quantum efficiency of ηqe=0.8 is possible with many (costly) optimizations. In current 1.4 μm consumer grade sensors the backside illumination (BSI) technology enables close to 100% fill factor.14 For color imaging, the spectral sensitivity is not without attenuation and peak quantum efficiencies of about ηqe0.5 are reported by OmniVision14 and Aptina.15 In scientific CMOS sensors, the combined sensor readout noise is reported as low as 1.3electrons/pixel16 and can thus be neglected among 1000 electrons. The combined assumption of ηlens=0.95, γff=0.95 and ηqe=0.5 leads to a minimum pixel size of dp,min=1.34μm. With mass-market sensors and additional noise,8 larger pixels are required.

These small pixels also reach another technological limit of decreasing full well capacity. For example Aptina reports15 C=5000 electrons, which leaves only a dynamic range of 51 from noise visibility5 to overexposure. As a result, most of the image will still look noisy. However, this is a technological challenge that could be addressed with multiple readouts during the exposure.17

Another limitation comes with optical diffraction. Even in ideal optics the achievable resolution of a camera system is limited. The Sparrow criterion suggests3 that there is no gain in resolution below a critical pixel size of dp,crit=λ2·f/D. For our example of f/D=2.8 and λ=555nm, we obtain dp,crit=0.78μm. Achieving this limit, however, is challenging, especially in the off-axis field, and leads to expensive optics. A further decrease in aperture requires a dramatic increase of the technological efforts and smaller tolerances for optics manufacturers.

6.

Conclusion

In our photometric analysis, we discuss the number of photons per pixel. With small pixels the image quality is limited by shot noise, and for indoor scenarios the current video cameras are surprisingly close to this fundamental limit. We estimate that even with ideal technology, a pixel size below dp=0.9μm will not capture enough light to generate visually pleasing videos any more. Current technology is far from perfect and with optimistic assumptions, the limit at dp=1.34μm is close to current sensors. However, for other imaging scenarios like outdoor daylight still photography, there is plenty of room at the bottom.

References

1. 

R. Fontaine, “A review of the 1.4 um pixel generation,” in Int. Image Sensor Workshop (IISW), Hokkaido, Japan (2011).Google Scholar

2. 

J. B. JonasU. SchneiderG. O. H. Naumann, “Count and density of human retinal photoreceptors,” Graefes Arch. Clin. Exp. Ophthalmol. 230, 505–510 (1992).GACODL0721-832Xhttp://dx.doi.org/10.1007/BF00181769Google Scholar

3. 

J. Goodman, Introduction to Fourier Optics, Roberts & Company Publishers, Englewood, Colorado, USA (2005).Google Scholar

4. 

J. Farrellet al., “A simulation tool for evaluating digital camera image quality,” in SPIE Electron. Imag.—Image Quality and System Performance 5294, 124–131 (2004).http://dx.doi.org/10.1117/12.537474Google Scholar

5. 

F. XiaoJ. FarrellB. Wandell, “Psychophysical thresholds and digital camera sensitivity: the thousand photon limit,” in SPIE Electron. Imag.—Digital Photography 5678, 75–84 (2005).http://dx.doi.org/10.1117/12.587468Google Scholar

6. 

R. Kingslake, Optical System Design, Academic Press, London 1 (Oct. 1983).Google Scholar

7. 

B. Fowleret al., “A method for estimating quantum efficiency for CMOS image sensors,” in SPIE Electron. Imag.—Solid State Sensor Arrays: Development and Applications II 3301, 178–185 (1998).http://dx.doi.org/10.1117/12.304561Google Scholar

8. 

R. Gowet al., “A comprehensive tool for modeling CMOS image-sensor-noise performance,” IEEE Trans. Electron. Devices 54, 1321–1329 (June 2007).IETDAI0018-9383http://dx.doi.org/10.1109/TED.2007.896718Google Scholar

9. 

W. Smith, Modern Optical Engineering: The Design of Optical Systems, Tata McGraw-Hill Education, Englewood, Colorado, USA (1990).Google Scholar

10. 

P. Giacomo, “News from the BIPM: resolution 3—definition of the candela,” Metrologia 16(1), 55–61 (1980).MTRGAU0026-1394http://dx.doi.org/10.1088/0026-1394/16/1/008Google Scholar

11. 

L. Sharpeet al., “A luminous efficiency function, V*(λ), for daylight adaptation,” J. Vision 5(11), 948–968 (2005).1534-7362http://dx.doi.org/10.1167/5.11.3Google Scholar

12. 

A. StockmanL. Sharpe, “The spectral sensitivities of the middle-and long-wavelength-sensitive cones derived from measurements in observers of known genotype,” Vis. Res. 40(13), 1711–1737 (2000).VISRAM0042-6989http://dx.doi.org/10.1016/S0042-6989(00)00021-3Google Scholar

13. 

R. Hannebaueret al., “Optimizing quantum efficiency in a stacked CMOS sensor,” in SPIE Electron. Imag.—Sensors, Cameras, and Systems for Industrial, Scientific, and Consumer Applications XII 7875(1), 787505 (2011).http://dx.doi.org/10.1117/12.873610Google Scholar

14. 

H. Rhodeset al., “The mass production of second generation 65 nm BSI CMOS image sensors,” in Int. Image Sensor Workshop (IISW), International Image Sensor Society (IISS) (2011).Google Scholar

15. 

G. Agranovet al., “Pixel continues to shrink … pixel development for novel CMOS image sensors: a review of the 1.4 um pixel generation,” in Int. Image Sensor Workshop (IISW), International Image Sensor Society (IISS) (2011).Google Scholar

16. 

B. Fowleret al., “A 5.5 mpixel 100frames/sec wide dynamic range low noise CMOS image sensor for scientific applications,” in SPIE Electron. Imag.—Sensors, Cameras, and Systems for Industrial/Scientific Applications XI 7536, 753607 (Jan. 2010).http://dx.doi.org/10.1117/12.846975Google Scholar

17. 

M. Schöberlet al., “Digital neutral density filter for moving picture cameras,” in SPIE Electron. Imag.—Computational Imag. VIII 7533, 75330L (January 2010).http://dx.doi.org/10.1117/12.838833Google Scholar

Michael Schöberl, André Kaup, Andreas Brückner, Siegfried Fößel, "Photometric limits for digital camera systems," Journal of Electronic Imaging 21(2), 020501 (15 June 2012). http://dx.doi.org/10.1117/1.JEI.21.2.020501
Submission: Received ; Accepted
JOURNAL ARTICLE
4 PAGES


SHARE
KEYWORDS
Signal to noise ratio

Sensors

Quantum efficiency

Cameras

Photons

Imaging systems

Digital cameras

RELATED CONTENT

Performance-based-CID-imaging-past-present-and-future...
Proceedings of SPIE (August 27 2008)
Digital-Method-To-Evaluate-The-Noise-Of-X-Ray-Image...
Proceedings of SPIE (December 26 1979)
Real-time-imaging-detectors-for-portal-imaging
Proceedings of SPIE (December 16 1993)
A-CMOS-camera-for-pyramid-wavefront-sensors
Proceedings of SPIE (July 15 2008)
Evaluation-of-a-backside-illuminated-ISIS
Proceedings of SPIE (February 10 2009)

Back to Top