Translator Disclaimer
Open Access Presentation + Paper
10 May 2017 Integral imaging with Fourier-plane recording
Author Affiliations +
Abstract
Integral Imaging is well known for its capability of recording both the spatial and the angular information of threedimensional (3D) scenes. Based on such an idea, the plenoptic concept has been developed in the past two decades, and therefore a new camera has been designed with the capacity of capturing the spatial-angular information with a single sensor and after a single shot. However, the classical plenoptic design presents two drawbacks, one is the oblique recording made by external microlenses. Other is loss of information due to diffraction effects. In this contribution report a change in the paradigm and propose the combination of telecentric architecture and Fourier-plane recording. This new capture geometry permits substantial improvements in resolution, depth of field and computation time
Conference Presentation

In 1908 Gabriel Lippmann reported a new imaging technique for the capture of dense perspective information of 3D scenes [1]. The technique was named as Integral Photography (IP), and was based in inserting an array of microlenses in front of a photographic film. In this way, a collection of small elemental images, each from a different point of view, could be obtained from 3D objects. Later, in 1992, Adelson and Wang [2] introduced the concept of plenoptic function, which describes the radiance of each luminous ray of light as a function of the angle and position in a given point of its trajectory. The IP was applied to microscopy, for the first time, by Jang and Javidi [3], but only with the purpose of displaying microscopic images. It is remarkable the contribution made by Levoy et al. [4] who adapted to microscopy the plenoptic concept. Very recently, the name integral microscopy (iMic) has been coined to refer to this technique. The improvement of the resolution of integral microscopes has aimed many research efforts. In this sense, are noticea-ble the application of ad-hoc deconvolution tools, the use of 4D interpolation techniques, or, more recently [5], the application of physical interpolation based in time multiplexing. The main drawback of iMics is their poor resolution. Factors that favor the loss of resolution are the limited number of pixels, the vignetting in the collected microimages and the comparable sizes of image PSF and microlens-array pitch.

To avoid the later drawbacks, in this contribution we report advances in a novel a novel architecture in which the microlens array is placed at the far-field plane, which in a microscope is just at the aperture stop of the microscope objective. This new realization of the plenoptic architecture is named as Far-field integral Microscopy (FiMic) or alter-natively as Fourier-domain integral Microscopy. As we explain below, The FiMic has some apparent advantages over the iMic: First is that it outputs directly the perspective views. Second is that there is no conflict between the micro-scope PSF and the microlenses sizes, and thus, no extra resolution is lost due to such conflict. Third advantage is the reduction in vignetted pixels. Final advantage is gain in compactness. A preliminary realization of FiMic was reported in [6].

Let us start by showing the typical scheme of an integral microscope, which can be built by performing some minor changes into an optical microscope, which we call hereafter as the native microscope (see Fig. 1). Schematically, optical microscope is transformed into an integral microscope by simply inserting an adequate microlens array (MLA) at the image plane (which is conjugated with the Reference Object Plane −ROP), and displacing axially the CCD, so that it is conjugated with the aperture stop of the objective.

Fig.1.

Schematic layout of an integral microscope.

00163_psisdg10219_102190b_page_1_1.jpg

In order to understand better the characteristics of an iMic, let us consider the parameters of a particular microscopy setup. Let us then consider that we are using a microscope objective (MO) of magnification M = 20 and numerical aperture NA = 0.4. Since the objective is designed for a tube lens of fL = 180 mm, the focal length of the MO is fob = 9.0 mm. Another important parameter is the exit pupil, which is inside the MO, and has diameter ϕEP = 7.1 mm. From these parameters we can calculate the resolution limit of the native microscope

00163_psisdg10219_102190b_page_2_1.jpg

which, considering a central wavelength λ = 0.55 μm, gives to ρnat = 0.84 μm at the object plane and ρnat = 16.8 μm at the image plane.

Next step is the selection of the optimum MLA. As well known, the resolution limit of calculated views and of rendered depth images is determined by the MLA pitch. Therefore, the smaller the pitch the better the resolution. However, smaller pitch implies stronger diffraction effects. So, in order to minimize such diffraction effects at the MLA plane, and therefore make the ray-optics approach reliable, the pitch should be at least five times larger than diffraction spot. So, an MLA with pitch p = 80 μm is adequate. On the other hand, and in order to make an efficient use of the pixels of the sensor, there must be a good matching between the NA and the microlenses f-number. Specifically, the matching relation is

00163_psisdg10219_102190b_page_2_2.jpg

In our case, the optimum f# = 25. About the sensor, there is not a big constraint in the selection of pixel size. According to Nyquist, sizes smaller than ρnat/2 = 8.4 μm are acceptable.

Even though the parameters of the iMic were optimized, there are still some remaining important problems. The problems are:

  • (a) At the sensor the diffraction spot is only 5 times smaller than the microimage. So, the angular information is captured with very poor resolution;

  • (b) The resolution of views and depth images is determined by the MLA pitch, which has a limit that cannot be overcome. In our case ρview = 5ρnat = 4.2 μm;

  • (c) The depth of field is short, and determined by the NA. In our case 00163_psisdg10219_102190b_page_2_3.jpg

  • (d) The views are not acquired directly, but have to be calculated;

  • (e) The microscope is not compact, but an elongated system.

In order to lessen the importance of theses problems, we have proposed a new paradigm in integral microscopy. In our proposal we remove the tube lens, and place directly the MLA at the aperture stop of the MO. Note however that in most commercial MO the aperture plane is not accessible. Even its conjugate, the exit pupil, is neither accessible. Instead it is virtual and placed within the MO. For this reason it is usually necessary the use of a telecentric relay system, as shown in Fig. 2.

Fig.2.

Schematic layout of a far-field integral microscope. In this case the images provided by the microlenses (which we will name as elemental images), are directly perspective views.

00163_psisdg10219_102190b_page_2_4.jpg

Again, in order to understand better the characteristics of FiMic, we consider the parameters of a particular microscopy setup. We use again the same MO (M = 20, NA = 0.4, fob = 9.0 mm, and ϕEP = 7.1 mm). Another parameter, which now is of great interest, is the angular field number (AFN). The AFN is defined as quotient between the aberration-free field of view (FOV) of the MO, and its focal length;

00163_psisdg10219_102190b_page_3_1.jpg

In our case, AFN = 7.8. Although the lateral (and angular) magnification of the relay system can play an important role in the fine adjustment of the microscope, in this example we consider that it has unit magnification and is formed by two identical lenses of focal length f1 = f2 = 50 mm.

Again, the next step is the selection of the optimum MLA. In this case, in order to avoid overlapping and of making an efficient use of the pixels of the sensor, the constraint condition is that the f-number of the microlenses should be equal than the AFN of the objective. The case in which AFN is slightly bigger than the f-number is also acceptable, since the field stop can be used to adjust the fields. Other constraints are: (a) A, relatively, high number of microlenses is needed in order to obtain good angular sampling; (b) Large apertures are needed in order to obtain elemental images with good resolution; and (c) The resolution limit of perspective views and depth rendered images is determined by diffraction, but also by the focal lengths of the elements in the way

00163_psisdg10219_102190b_page_3_2.jpg

In the above equation, ρFiMic is the resolution limit at the object space, ρEI the resolution limit at the elemental images, and m is the numbers of microlenses along a diameter of the MLA.

Taking into account these constraints, the optimum MLA are arranged in hexagonal way and composed by seven or by nineteen microlenses, see Fig. 3.

Fig. 3.

Optimum microlens arrays for FiMic.

00163_psisdg10219_102190b_page_3_3.jpg

In the case of a MLA composed by nineteen microlenses, and taking into account the parameters of the MO, the pitch should be about p = 1.4 mm, and the f-number about f# = 7. Note that in fact small changes in the value of f# only affect the final FOV of the system, but neither the resolution nor the DOF. The EIs are circular with 1.4 mm in diameter, and in which the diffraction spot has radius ρEI = 4.7 μm. Note that, contrarily to the case of the iMic, now the diffraction spot 335 times smaller than the EI, and therefore the EIs can be captured sharply. Concerning the CCD, according to Nyquist criterion, an optimum pixel size is about δ = 2.3 μm, which is very close to standard commercial values.

With this system it is possible then to capture directly nineteen orthographic views with resolution limit ρFiMiC = 4.2 μm. Depending on the depth position, rendered depth images can have resolution of up to ρRec = 2.1 μm [7]. Finally, the DOF is also enlarged, since it is given by the formula

00163_psisdg10219_102190b_page_3_4.jpg

In the case of the MLA composed by seven microlenses the FiMic parameters are: p = 2.3 mm, f# = 7, ρEI = 4.7 μm, δ = 2.3 μm, ρFiMic = 2.5 μm, ρRec = 1.3 μm and DOF = 31.5 μm.

In summary, we present here a new design paradigm in integral microscopy. The main advantages of the new design are:

  • (a) Extra diffraction effects are avoided. Of course microscopy is a technique that depends strongly on diffraction. We do not avoid this intrinsic dependence. But avoid the introduction of extra limitations due to diffraction.

  • (b) Perspective views are captured directly without any need of computer processing. Thus, great computation time is saved.

  • (c) Perspective views have good resolution (1/3 or 1/5 of native resolution) and very good DOF (9 or 25 times the native DOF).

  • (d) The system is very compact and can work in brightfield and in fluorescence mode.

ACKNOWLEDGEMENTS

This work was supported in by the Plan Nacional I+D+I, under the grant DPI2015-66458-C2-1R, Ministerio de Economía y Competitividad, Spain. We also acknowledge the support from the Generalitat Valenciana (GVA), Spain, (grant PROMETEOII/2014/072). A. Llavador acknowledges a predoctoral contract from University of Valencia (UV-INV-PREDOC13-110484). E. Sanchez-Ortiga acknowledges a postdoctoral contract from GVA (APOSTD/2015/094). J. Sola-Pikabea acknowledges a predoctoral contract from GVA (ACIF/2016/296). G. Scrofani acknowledges a predoctoral contract from EU H2020 program under MSCA grant 676401.

References

[1] 

G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys., 7 821 –825 (1908). Google Scholar

[2] 

E. H. Adelson and J. Y. A. Wang, “Single lens stereo with plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell., 14 99 –106 (1992). https://doi.org/10.1109/34.121783 Google Scholar

[3] 

J. S. Jang and B. Javidi, “Three-dimensional integral imaging of micro-objects,” Opt. Lett., 29 1230 –1232 (2004). https://doi.org/10.1364/OL.29.001230 Google Scholar

[4] 

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light Field Microscopy,” ACM Trans. Graph, 25 934 –934 (2006). https://doi.org/10.1145/1141911 Google Scholar

[5] 

A. Llavador, E. Sánchez-Ortiga, J.C. Barreiro, G. Saavedra, and M. Martínez-Corral, “Resolution enhancement in integral microscopy by physical interpolation,” Biomed. Opt. Express, 6 2854 –2863 (2015). https://doi.org/10.1364/BOE.6.002854 Google Scholar

[6] 

A. Llavador, J. Sola-Pikabea, G. Saavedra, B. Javidi, and M. Martinez-Corral, “Resolution improvements in integral microscopy with Fourier plane recording,” Opt. Express, 18 20792 –20798 (2016). https://doi.org/10.1364/OE.24.020792 Google Scholar

[7] 

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martínez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Display Technol., 9 37 –43 (2013). https://doi.org/10.1109/JDT.2012.2225018 Google Scholar
© (2017) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
M. Martínez-Corral, J. C. Barreiro, A. Llavador, E. Sánchez-Ortiga, J. Sola-Pikabea, G. Scrofani, and G. Saavedra "Integral imaging with Fourier-plane recording", Proc. SPIE 10219, Three-Dimensional Imaging, Visualization, and Display 2017, 102190B (10 May 2017); https://doi.org/10.1117/12.2263520
PROCEEDINGS
4 PAGES + PRESENTATION

SHARE
Advertisement
Advertisement
RELATED CONTENT

Correlation plenoptic imaging
Proceedings of SPIE (June 26 2017)
3D integral microscopy based in far-field detection
Proceedings of SPIE (June 19 2018)
Hybrid open top light sheet microscopy for 3D imaging of...
Proceedings of SPIE (January 01 1900)
Overcoming sensor constraints in 3D integral imaging
Proceedings of SPIE (October 26 2005)

Back to Top