Translator Disclaimer
26 October 2020 Aliasing mitigation in optical microscopy of dynamic biological samples by use of temporally modulated color illumination and a standard RGB camera
Author Affiliations +
Abstract

Significance: Despite recent developments in microscopy, temporal aliasing can arise when imaging dynamic samples. Modern sampling frameworks, such as generalized sampling, mitigate aliasing but require measurement of temporally overlapping and potentially negative-valued inner products. Conventional cameras cannot collect these directly as they operate sequentially and are only sensitive to light intensity.

Aim: We aim to mitigate aliasing in microscopy of dynamic monochrome samples by implementing generalized sampling via the use of a color camera and modulated color illumination.

Approach: We solve the overlap problem by spectrally multiplexing the acquisitions and using (positive) B-spline segments as projection kernels. Reconstruction involves spectral unmixing and inverse filtering. We implemented this method using a color LED illuminator. We evaluated its performance by imaging a rotating grid and its applicability by imaging the beating zebrafish embryo heart in transmission and light-sheet microscopes.

Results: Compared to stroboscopic imaging, our method mitigates aliasing with performance improving as the projection order increases. The approach can be implemented in conventional microscopes but is limited by the number of available LED colors and camera channels.

Conclusions: Generalized sampling can be implemented via color modulation in microscopy to mitigate temporal aliasing. The simple hardware requirements could make it applicable to other optical imaging modalities.

1.

Introduction

Observing phenomena in live biological samples in microscopy requires sufficient time resolution.1 In addition to the development of faster and more sensitive cameras and clever pixel rebinning methods,2 various sensing and computational approaches to increase the temporal resolution of microscopes have been proposed. Some rely on multiple observations of a signal36 or make clever use of the signal structure itself, e.g., its sparsity in a known basis79 or its repeatable nature.10,11 The ability to modulate the illumination rapidly in a controlled and cost-effective way (in particular, with LED-based illuminators12) also opens the way for promising methods. For instance, short light pulses (stroboscopy) have been used to reduce motion blur,13 while the fluttered shutter principle14,15 uses a pseudorandom temporal illumination sequence to computationally improve the temporal resolution.

Despite the above developments, many imaging systems still rely on direct image acquisition, which is vulnerable to aliasing if the imaged signal contains frequencies higher than the Nyquist frequency, since perfect low-pass filters cannot be implemented in practice. Generalized sampling16 offers a framework to implement sampling operations that relax the need for ideal filters. Applications that build upon this framework have been proposed in optics17 or for sample signals with a finite rate of innovation on multiple channels.18

Due to the shift-invariant nature of generalized sampling, the implementation of prefilters in time can be problematic because of the temporal overlap of the inner product kernels. Indeed, if the prefilter used is longer than a unit of time (sampling interval), several inner products must be carried out simultaneously, which requires a multiplexed acquisition approach. Moreover, in standard (incoherent light) optical imaging applications, only the intensity of the light, which is always positive, can be measured. While modulation is possible by illuminating the sample with a variable intensity over time, this illumination suffers from the same positivity limitations (only positive illumination functions can be considered). In this paper, we present an approach to overcome the above positivity and multiplexing limitations to carry out generalized sampling in the context of optical microscopy imaging. We propose to use active multicolor illumination and a color camera for collection, allowing spectral multiplexing and ideal prefiltering of the signal. Specifically, we modulate the illumination signal over time with independent signals in different color channels of the illumination lamps, which produces modulated signals whose integration by a camera shutter can be converted to the inner product between the signal and the prefilter kernel. Following recovery of these coefficients and reconstruction of the signal in the projection basis’s dual basis, we obtain samples of the incoming signal, projected on the space spanned by a shift-invariant B-spline basis.

This paper is based on our previous paper introducing generalized temporal sampling imaging for microscopy.19 The rest of the paper is organized as follows. In Sec. 2, we provide a formal description of our problem. In Sec. 3, we derive our method. In Sec. 4, we illustrate our approach on both synthetic signals and from data collected on a transmission wide-field microscope. Finally, we conclude in Sec. 5.

2.

Problem Statement

We consider a continuous-time signal f(t) for which we want to estimate the least-squares approximation f˜(t) in the shift-invariant space spanned by B-splines of degree n, V={βn(·k),kZ}, which can be obtained via a biorthogonal projection16

Eq. (1)

f˜(t)=kZf,β°n(·k)c[k]βn(tk),
with

Eq. (2)

f,g=f(t)g(t)dt
and where β°n(t) is the dual B-spline,20 a function which, in addition to satisfying,

Eq. (3)

βn(·),β°n(·k)=δ[k],
also spans the subspace V (and is, therefore, unique).

A practical system should thus provide the measurements ck, which correspond to weighted integrals of the input signals. There are two issues when implementing this. First, the β°n has infinite support and hence overlap in time (except for degree n=0), which requires some way of splitting the input signal to perform the inner products in parallel, and second, the signals in this setting are light intensities, which have to be positive.

To implement the projection settings, we consider a conventional imaging system with a camera that has C color channels (e.g., an RGB pixel of a color camera corresponds to C=3). We further assume that the system has L light sources that uniformly illuminate the scene, each with a different, yet possibly overlapping, color spectrum. The intensity of each light source can be independently controlled as a function of time. These functions can only be positive (as light intensities are necessarily positive) and their duration should be less than the integration time of each of the camera’s frames. We model the imaging system by taking into account the crosstalk that arises from the use of broadband light sources and wide camera (RGB) filters. For a single pixel at time k, we simultaneously measure the intensities yc[k] in channels c=1,,C

Eq. (4)

(y1[k]yC[k])=(γ1,1γ1,Ld1γC,1γC,LdC)(a1[k]aL[k]1)
or in condensed form

Eq. (5)

y=Γa,
where the matrix Γ contains the crosstalk mixing terms (including an affine offset) and the coefficients a[k] are the inner products

Eq. (6)

a[k]=f,s(·k),
where s(t) are the positive illumination functions, whose support covers the sensor exposure interval.

With this formulation, the problem of recovering the samples c[k] in Eq. (1) from measurements yc[k] can be broken down into the following subproblems:

  • 1. Find suitable illumination functions s (in particular, positive and with a finite support) such that the sequence c[k] can be derived from the multichannel sequence a[k].

  • 2. Determine the coefficients c[k] from the coefficients a[k] (via inverse filtering).

  • 3. Determine the coefficients a[k] from the yc[k] (via spectral unmixing).

We detail these steps in the section below.

3.

Methods

3.1.

Multicolor B-Spline Segments as Prefilter Kernels

Our goal is to compute the inner products in Eq. (1) via active illumination by appropriately choosing illumination functions s(tk). The challenge of this task is that we cannot use s(tk)=β°n(tk) directly (which would be the natural choice) because the dual of a B-spline is not positive for all t (except when n=0) and because s(t) a light intensity, it must be positive. We work around this problem using an equivalent representation of the projection described in Eq. (1) and shown in Fig. 1, by switching the role of the dual bases20 namely,

Eq. (7)

f˜(t)=kZf,β°n(·k)βn(tk)=kZf,βn(·k)β°n(tk).

Fig. 1

Equivalent basic and dual B-spline representations of the same signal for degrees (a), (b)n=1 and (c), (d) n=2. Notice that both represented signals (in blue in all plots) are equivalent, i.e., kc[k]βn(tk)=kc°[k]β°n(tk), while the basic B-spline representation uses functions of finite support.

JBO_25_10_106505_f001.png

Specifically, the coefficients in the dual B-spline basis are given as

Eq. (8)

c°[k]=βn(·k),f=n+12n+12βn(tk)f(t)dt,
where we note that all involved functions are positive and, since the B-spline has a finite support, the inner-product can be computed over a finite interval. Nevertheless, since the support of B-splines of degree n is n+1, we have n+1 shifted B-splines that overlap in the signal representation at any given time (see Fig. 1) making the sequential computation of the c°[k] problematic.

In order to still acquire several inner products simultaneously, we spectrally multiplex the measurements according to Eq. (4) by splitting each B-spline into n+1 regions, which gives the following illumination functions s(t). For n illumination sources (=1,,n); for n=0, we have

Eq. (9)

s1(t)=β0(t),
for n=1, we have

Eq. (10)

s1(t)={β1(t)=1t,0<t<10,otherwise,

Eq. (11)

s2(t)={β1(t1)=t,0<t<10,otherwise,
and for n=2, we have

Eq. (12)

s1(t)={β2(t+1)=12(12t)21/2<t<1/20,otherwise,

Eq. (13)

s2(t)={β2(t)=34t2,1/2<t<1/20,otherwise,

Eq. (14)

s3(t)={β2(t1)=12(12+t)21/2<t<1/20,otherwise,
where

Eq. (15)

β2(t)={12(32|t|)2,12<|t|<3234t2,|t|120,otherwise.

Figure 2 shows the illumination functions over three consecutive acquired frames for stroboscopic illumination and splines of degrees n=0, n=1, and n=2. The color of the line corresponds to the color of the illumination. Notice that to reconstruct c°[k] we have to combine measurements over multiple frames (see the bold lines in Fig. 2 for n=1 and n=2).

Fig. 2

Illumination functions s(tk) over three frames for stroboscopic imaging (a) n=0, n=1, and (c) n=2. The color of the line corresponds to the color of the illumination light. For (c), (d) n=1 and n=2, multiple frames are involved to compute a single coefficient c°[k] from the coefficients a.

JBO_25_10_106505_f002.png

The red illumination comes from light source 1 (a1), the blue illumination comes from light source 2 (a2), and the green illumination from light source 3 (a3). For degrees n=0,1, and 2, the coefficients c°[k] can be recovered from a[k] as described below (assuming mirror boundary conditions).

Degree n=0:

Eq. (16)

c°[k]=a1[k].

Degree n=1:

Eq. (17)

c°[k]={a1[k]+a2[k+1],0k<K12a1[K1],k=K1.

Degree n=2:

Eq. (18)

c°[k]={a1[k1]+a3[k]+a2[k+1],1k<K2a2[0]+a3[0]+a2[1],k=0a1[K1]+a1[K1]+a1[K2],k=K1.

3.2.

Spectral Unmixing

The spectral unmixing procedure is similar to the setting described in Jaques et al.21 Specifically, given the measurements y and having built d and Γ from calibration, we recover the vector a by solving the minimization problem

Eq. (19)

a=minaydΓa22.

3.3.

Converting to Samples

We obtain the basic B-spline representation through filtering of the coefficients in the dual-spline representation20

Eq. (20)

c[k]=(b2n+1)1*c°[k].

We can further obtain samples by carrying out interpolation

Eq. (21)

f˜[k]=f˜(t)|t=k=c*b1n[k],
where

Eq. (22)

b1n[k]=βn(t)|t=k.

4.

Experiments

4.1.

Reconstructions from Synthetic Data

We conducted an experiment to illustrate the impact of the chosen sampling basis. We generated a one-dimensional (1-D) signal containing shifted B-splines of various degrees, shown in Fig. 3. We then simulated active sampling using B-splines of degrees 0 to 2 as prefilters. Figure 3 shows the reconstructions obtained by sampling the signal on the top left of the figure. We observe that the B-splines in the sampled signal are perfectly reconstructed when the prefilter is of the same degree as the B-spline. We also see overshooting and ringing after sharp transitions, for degrees n1

β0(t5)+β1(t10)+β2(t15)+β3(t20).

Fig. 3

Shifted B-splines of degrees 0, 1, and 2. We observe that each B-spline is perfectly reconstructed when the sampling is done with B-splines of the same degree, i.e., the B-spline of degree 0 in (b) is perfectly sampled, the B-spline of degree 1 in (c) is perfectly sampled, and the B-spline of degree 2 in (d) is perfectly sampled.

JBO_25_10_106505_f003.png

We set out to investigate the effect of our method in the presence of high frequencies, which we compared to a stroboscopic imaging system. We generated a 1-D temporal chirp signal and simulated sampling with B-splines of degree 2 as well as stroboscopic imaging. The signal and sampling simulations are shown in Fig. 4, where we notice that stroboscopic imaging is strongly subject to aliasing, while generalized sampling by projection on B-spline bases gracefully handles higher frequencies. Our chirp function is f(t)=1+sin(t235+π2)=1+sin(ϕ(t)), with the instantaneous ordinary frequency defined as dϕ(t)dt12π=t35π. In the simulation, our sampling period is 1, so the Nyquist frequency is 1/2, i.e., the Shannon–Nyquist sampling criterion is not respected anymore when t35π12t=35π255. In Fig. 4, we show the normalized energy of the error of reconstruction computed on the signal where the Shannon–Nyquist criterion is not respected (after the black vertical line on the plots). The stroboscopic reconstructions exhibit the highest error.

Fig. 4

Sampling simulations of the signal in (a) f(t)=1+sin(t235+π2). We simulated (b) stroboscopic imaging and generalized sampling with B-splines of (c) degrees 1 and (d) 2. The vertical black line shows time after which the Shannon–Nyquist criterion is no longer respected (frequency too high). For each plot, the reconstruction error energy is shown and stroboscopic has the higher error. Notice that (c), (d) the B-spline sampling reconstructions for high frequencies (t80) goes to the average of the signal, while (b) the stroboscopic imaging shows aliasing.

JBO_25_10_106505_f004.png

4.2.

Rotating Chirp

We used a printed black and white grid that we rotated using a stepper motor (Nema 14, Bipolar, Stepper Online, China) controlled by a microcontroller (Arduino Due, Arduino, Italy) through a power driver module (L298N dual H-bridge driver chip, Electronicmodule Store, China). We incrementally increased the grid’s rotation speed by steps, while acquiring images with either stroboscopic illumination or using the illumination functions shown in Fig. 2. The exposure time was set to E=200  ms. We then used the method presented in Sec. 3.1 to reconstruct image series and compared them to those obtained via stroboscopic imaging (Fig. 5). Figures 5(a)5(d) show acquired images using either stroboscopic imaging (a) or B-spline sampling with degrees 0 to 2 (b)–(d). Figures 5(e)5(h) show time profiles extracted from the reconstructed image series corresponding to the locations indicated by the symbols x, *, +, and o in (a), (b), (c), and (d), respectively. Figures 5(i)5(l) show reconstructed images when the grid was rotating at high speed. We can see strong temporal aliasing in (i) as the grid appears to be almost static over consecutive frames, although it has undergone multiple rotations. In Figs. 5(j)5(l), the rotating grid takes the aspect of a uniform gray disk, with minor intensity variations where aliasing is slightly visible. This experiment shows that the sampling scheme presented in Sec. 3.1 does not allow for perfect sampling, which would be free of aliasing and motion blur. Nevertheless, the prefilter is optimal for the sampling in the bases in which we sample (project) the imaged signal, hence the reconstructions of Figs. 5(j)5(l) are the optimal representations of the signal in our chosen B-spline bases. In other words, even if our reconstructions are not perfect, they are more reliable than that of Fig. 5(i). Similar to the experiment in Sec. 4.1, we can see that the reconstructions in Figs. 5(f)5(h) at high speed (right part of the plot, where t>8  s) tend to approximate the average between black and white, which is around 0.2.

Fig. 5

Rotating grid imaged using different sampling methods. (a) Still frame of a rotating grid (arrow indicates direction of rotation, rotation speed was increased over time) under stroboscopic light. (b)–(d) Images of the same grid as in (a), rotating at the same speed, but acquired using B-splines as prefilters, with degrees (b) 0, (c) 1, and (d) 2. (e)–(h) Time profile extracted from the reconstructed image series corresponding to the locations indicated by the symbols x, *, +, and o in (a)–(d), respectively (compare to similar experiment on synthetic data in Fig. 4). Notice that in (e) (stroboscopic case), for t>6, the plot oscillates between the maximal and minimal values (corresponding to white and black in the image) while in (f)–(h) (generalized sampling cases), the values are close to average intensity (gray in the images). (i) Images acquired with the stroboscopic illumination when the grid was rotating fast. Strong aliasing is visible, as between consecutive frames the grid appears to have moved very little, while it has made more than one rotation between each frame. (j)–(l) Reconstructions with B-spline prefilters of degree (j) 0, (k) 1, and (l) 2. Images in (j)–(l) show little to no aliasing: the rotating grid becomes a uniform gray disk, which is consistent with the simulation in Fig. 4. Full movie in Video 1. Scale bars: 2 cm (Video 1 [URL: https://doi.org/10.1117/1.JBO.25.10.106505.1], MP4, 13 MB).

JBO_25_10_106505_f005.png

Since the ground-truth is not known, we use the energy between the signal and the average gray value between white and black as a proxy for the error at high frequencies, that is, to the right of the vertical line in Figs. 5(e)5(h), similar to the experiment in Sec. 4.1. As noted by other authors, quantifying aliasing is not a trivial task22,23 and we can only provide a proxy to quantify the error. We see that stroboscopic imaging exhibits the highest error. Also, the two black arrows in Fig. 5(e) show aliasing where the grid appears to be rotating slowly, while it undergoes multiple rotations between two consecutive acquisitions.

4.3.

Microscopy

We set out to investigate if our method could be implemented in optical microscopy. Specifically, we considered bright field and fluorescence (light-sheet) microscopy. We further investigated the feasibility of using this method for in vivo imaging of dynamic processes, specifically, to image the beating heart of zebrafish embryos.

4.3.1.

Hardware and parameters setup

For bright-field microscopy (experiments in Sec. 4.3.2), we implemented the illumination scheme with commonly available and cost-effective hardware. We assembled a light source using a 6-LED chip (SLS Lighting RGBWA+UV, Aliexpress, China). We drove the red (λ620  nm), green (λ525  nm), and blue (λ465  nm) LEDs via a microcontroller (Arduino Uno, Arduino, Italy), which we programmed to generate the illumination time-pattern shown in Fig. 2, individually controlling each color. For the LED and camera synchronization, the microcontroller monitored the flash trigger output of the camera. Whenever the trigger signal transitions from low to high state, the microcontroller starts the time sequence of the LEDs for the frame about to be recorded. The LEDs were directly powered by the controller’s outputs without additional power amplification of the signal.

For fluorescence microscopy (experiments in Sec. 4.3.3), we used an implementation of the OpenSPIM light-sheet microscope,24 with two lasers (Stradus, Vortran Laser Technology) of wavelengths 488 and 561 nm to generate the excitation illumination light sheet. Again, we used a microcontroller (Arduino Uno) to modulate the laser intensities over time, using pulse-width modulation control on their fast ON/OFF electrical connection to generate the illumination functions in Fig. 2 up to degree 1 (the highest degree achievable with only two lasers).

For both the bright-field and fluorescence experiments, we used a CMOS color camera (Thorlabs DCC3240C, Thorlabs, Germany) with 1280×1024  pixels and a standard RGGB-Bayer filter pattern. We attached the camera to the camera port of our microscope (for both transmission and light-sheet microscopy) consisting of a 20× Olympus water dipping lens (Olympus Plan Fluorite UMPLFLN 20×W) combined with a 180-mm tube lens (Olympus U-TLU-1-2) and terminated by a 0.5× zoom lens (Olympus U-TV0.5XC-3).

For imaging, we embedded 3 dpf zebrafish larvae, anesthetized with 0.1% tricaine (ethyl 3-aminobenzoate methanesulfonate salt, Sigma), in low melting agarose.

4.3.2.

Generalized sampling of the beating heart of the zebrafish under transmission microscopy

To illustrate the applicability of our method for biological bright field microscopy, we imaged the beating heart of the zebrafish larva with transmission illumination. We first acquired images of the beating heart using stroboscopic imaging then repeated the acquisition using the generalized sampling method from Sec. 3 with degrees 0, 1, and 2. The exposure time was set to E=60  ms. Figure 6 shows three consecutive images either acquired with a strobed illumination [Fig. 6(a)] or using temporal generalized sampling [Figs. 6(b)6(d)].

Fig. 6

Generalized sampling of a 3-dpf zebrafish beating heart under transmission microscopy. (a) Three consecutive frames of stroboscopic imaging. (b)–(d) Three consecutive reconstructed frames using our method in Sec. 3 for B-spline sampling of degrees (b) 0, (c) 1, and (d) 2. There are no obvious differences between (a) to (d), (a) is slightly sharper than (b) to (d) due to the short light pulse. Full movie in Video 2. Scale bar: 100  μm (Video 2 [URL: https://doi.org/10.1117/1.JBO.25.10.106505.2], MP4, 2 MB).

JBO_25_10_106505_f006.png

Although the strobed illumination [Fig. 6(a)] produces slightly sharper images, this approach is subject to aliasing at high frequencies. This means that even though the image appears sharper, one cannot trust the observed motions to be representative of the actual motion sequence of the heart. Although the images obtained with our proposed method [Figs. 6(b)6(d)] exhibit stronger motion blur than those obtained with strobed light, one can be confident that the perceived motion is accurate.

4.3.3.

Generalized sampling of the beating heart of the zebrafish on light-sheet fluorescent microscopy

We set out to investigate whether our generalized sampling method is applicable to fluorescence microscopy. To take advantage of the temporally modulated color illumination, we imaged a zebrafish that coexpresses ubiquitous cytoplasmic green fluorescent protein, EGFP, and red fluorescent protein, mCherry.25,26 Taking advantage of the fluorophores being colocalized, we simultaneously computed the inner products in Eq. (8) and used the method of Sec. 3 to perform temporal generalized sampling.

We acquired images of the beating heart of the zebrafish first using stroboscopic imaging then, via the generalized sampling method from Sec. 3, with degrees 0 and 1. The exposure time was set to E=60  ms. Since we had only two lasers and two fluorophores in our system, only generalized sampling up to degree 1 was possible.

Figure 7 shows three consecutive images acquired with either the strobed illumination [Fig. 6(a)] or our method [Figs. 6(b) and 6(c)].

Fig. 7

Generalized sampling of a 3-dpf zebrafish beating heart under light-sheet fluorescence microscopy. (a) Three consecutive frames of stroboscopic imaging. (b), (c) Three consecutive reconstructed frames using our method in Sec. 3 for B-spline sampling of degrees (b) 0 and 1 (c). There are no obvious differences between (a) to (c), while the reconstructions in (b) and (c) can be trusted more than (a). Full movie in Video 3. Scale bar: 100  μm (Video 3 [URL: https://doi.org/10.1117/1.JBO.25.10.106505.3], MP4, 1.2 MB).

JBO_25_10_106505_f007.png

Similar to the experiment in Sec. 4.3.2 for transmission microscopy, the strobed illumination produces slightly sharper images. However, the strobed image sequence is subject to stronger aliasing.

5.

Conclusion

We presented a method to perform temporal generalized sampling in optical microscopy. Our approach allows for the use of implementable prefilters that have finite temporal support and that verify the positivity constraint. Via our proposed spectral multiplexing approach, we could simultaneously compute multiple inner products and, after spectral unmixing, retrieve each individual inner product. Our method leverages the equivalence between basic and dual B-spline representations.20 Since our method follows the projection framework of generalized sampling, it offers the prospect of avoiding aliasing despite not using ideal prefilters.

In the experiments of Sec. 4.3, the beating heart of the zebrafish respects the Shannon–Nyquist criterion of our imaging system and lateral motion remains limited, hence our method produces results similar to those obtained via stroboscopic imaging. That is, in this particular case, the imaged sample does not exhibit aliasing when imaged with a stroboscopic imaging system with an image every 60 ms. However, our method increases the trust one can have in the captured videos since an insufficient frame-rate would have resulted in motion blur with our projected method, which would have been visible even in still frames. For biomedical experimentalists, the advantage of using our method is that sharp image sequences are more likely to be devoid of temporal aliasing. Also, while a possible strategy for mitigating the potential risk of aliasing during stroboscopic imaging could be to increase the duration of the pulse width until aliasing disappears (also introducing blurring), this approach would require to actively test for the presence of aliasing, which could be difficult to automatically carry out. Since our method relies on projecting the sampled signal in a predefined basis, the illumination functions are fixed, which may introduce more blurring than necessary in situations where the imaged frequencies are sufficiently low for a strobed approach (Fig. 2).

Higher degree B-splines have a higher approximation power.27,28 Therefore, as high a degree as possible would be preferable. The highest achievable degree, in our experimental setting with an RGB color camera, was 2 since we could compute no more than three simultaneous inner products. While this paper focused on implementing B-spline prefilters, our method of simultaneously computing multiple inner products could be extended to other sensing methods provided the modulation functions are positive and have finite support.

Our experimental implementation relies on a global shutter camera. That is, all pixels share the same integration time and no light is captured by the sensor during the readout (when the image is transferred from the camera to the computer). On the camera we used, this readout time is about 3 ms for images of resolution 500 × 500 pixels. This implies that the partition of unity16 condition is not strictly respected. Nevertheless, in practice, this did not visibly affect our method. Some rolling shutter cameras permit continuous exposure of the sensor, where lines are sequentially exposed and transferred to the computer, but they would require additional adaptation to take this exposure sequence into account.

Disclosures

The authors declare that there are no conflicts of interest related to this article.

Acknowledgments

The authors would like to thank A. Ernst and N. Mercader from the University of Bern, Switzerland, for providing us with the zebrafish used in the experiments in Sec. 4.3. This research was funded by the Swiss National Science Foundation, Grant No 200021-159227 “Computational Methods for Temporal Super-resolution Microscopy,” Grant No. 206021-164022 “Platform for Reproducible Acquisition, Processing, and Sharing of Dynamic, Multi-Modal Data,” and Grant No. 200020_179217, “Computational biomicroscopy: advanced image processing methods to quantify live biological systems.”

References

1. 

J. Vermot, S. E. Fraser and M. Liebling, “Fast fluorescence microscopy for imaging the dynamics of embryonic development,” HFSP J., 2 (3), 143 –155 (2008). https://doi.org/10.2976/1.2907579 HJFOA5 1955-2068 Google Scholar

2. 

G. Bub et al., “Temporal pixel multiplexing for simultaneous high-speed, high-resolution imaging,” Nat. Methods, 7 (3), 209 –211 (2010). https://doi.org/10.1038/nmeth.1429 1548-7091 Google Scholar

3. 

E. Shechtman, Y. Caspi and M. Irani, “Space-time super-resolution,” IEEE Trans. Pattern Anal. Mach. Intell., 27 (4), 531 –545 (2005). https://doi.org/10.1109/TPAMI.2005.85 ITPIDJ 0162-8828 Google Scholar

4. 

T. Li et al., “Space-time super-resolution with patch group cuts prior,” Signal Process. Image Commun., 30 147 –165 (2015). https://doi.org/10.1016/j.image.2014.10.007 Google Scholar

5. 

A. Agrawal et al., “Optimal coded sampling for temporal super-resolution,” in IEEE Comput. Soc. Conf. Comput. Vision and Pattern Recognit., 599 –606 (2010). https://doi.org/10.1109/CVPR.2010.5540161 Google Scholar

6. 

R. Pournaghi and X. Wu, “Coded acquisition of high frame rate video,” IEEE Trans. Image Process., 23 (12), 5670 –5682 (2014). https://doi.org/10.1109/TIP.2014.2368359 IIPRE4 1057-7149 Google Scholar

7. 

T.-H. Tsai et al., “Spectral-temporal compressive imaging,” Opt. Lett., 40 (17), 4054 –4057 (2015). https://doi.org/10.1364/OL.40.004054 OPLEDP 0146-9592 Google Scholar

8. 

R. Koller et al., “High spatio-temporal resolution video with compressed sensing,” Opt. Express, 23 (12), 15992 (2015). https://doi.org/10.1364/OE.23.015992 OPEXFF 1094-4087 Google Scholar

9. 

P. Llull et al., “Coded aperture compressive temporal imaging,” Opt. Express, 21 (9), 10526 –10545 (2013). https://doi.org/10.1364/OE.21.010526 OPEXFF 1094-4087 Google Scholar

10. 

K. G. Chan et al., “Simultaneous temporal superresolution and denoising for cardiac fluorescence microscopy,” IEEE Trans. Comput. Imaging, 2 (3), 348 –358 (2016). https://doi.org/10.1109/TCI.2016.2579606 Google Scholar

11. 

A. Veeraraghavan, D. Reddy and R. Raskar, “Coded strobing photography: compressive sensing of high speed periodic videos,” IEEE Trans. Pattern Anal. Mach. Intell., 33 671 –686 (2011). https://doi.org/10.1109/TPAMI.2010.87 ITPIDJ 0162-8828 Google Scholar

12. 

J. B. Bosse et al., “Open LED illuminator: a simple and inexpensive LED illuminator for fast multicolor particle tracking in neurons,” PLoS One, 10 e0143547 (2015). https://doi.org/10.1371/journal.pone.0143547 POLNCL 1932-6203 Google Scholar

13. 

D. W. Staudt et al., “High-resolution imaging of cardiomyocyte behavior reveals two distinct steps in ventricular trabeculation,” Development, 141 (3), 585 –593 (2014). https://doi.org/10.1242/dev.098632 Google Scholar

14. 

R. Raskar, A. Agrawal and J. Tumblin, “Coded exposure photography: motion deblurring using fluttered shutter,” ACM Trans. Graphics, 25 795 –804 (2006). https://doi.org/10.1145/1141911.1141957 ATGRDF 0730-0301 Google Scholar

15. 

S. S. Gorthi, D. Schaak and E. Schonbrun, “Fluorescence imaging of flowing cells using a temporally coded excitation,” Opt. Express, 21 5164 –5170 (2013). https://doi.org/10.1364/OE.21.005164 OPEXFF 1094-4087 Google Scholar

16. 

M. Unser, “Sampling—50 years after Shannon,” Proc. IEEE, 88 (4), 569 –587 (2000). https://doi.org/10.1109/5.843002 IEEPAD 0018-9219 Google Scholar

17. 

N. Chacko, M. Liebling and T. Blu, “Discretization of continuous convolution operators for accurate modeling of wave propagation in digital holography,” J. Opt. Soc. Am. A, 30 2012 –2020 (2013). https://doi.org/10.1364/JOSAA.30.002012 JOAOD6 0740-3232 Google Scholar

18. 

K. Gedalyahu, R. Tur and Y. C. Eldar, “Multichannel sampling of pulse streams at the rate of innovation,” IEEE Trans. Signal Process., 59 1491 –1504 (2011). https://doi.org/10.1109/TSP.2011.2105481 ITPRED 1053-587X Google Scholar

19. 

C. Jaques and M. Liebling, “Generalized temporal sampling with active illumination in optical microscopy,” Proc. SPIE, 11138 1113816 (2019). https://doi.org/10.1117/12.2529526 PSISDG 0277-786X Google Scholar

20. 

M. Unser, A. Aldroubi and M. Eden, “The L2 polynomial spline pyramid,” IEEE Trans. Pattern Anal. Mach. Intell., 15 (4), 364 –379 (1993). https://doi.org/10.1109/34.206956 ITPIDJ 0162-8828 Google Scholar

21. 

C. Jaques et al., “Temporal super-resolution microscopy using a hue-encoded shutter,” Biomed. Opt. Express, 10 4727 –4741 (2019). https://doi.org/10.1364/BOE.10.004727 BOEICL 2156-7085 Google Scholar

22. 

C. Loebich et al., “Digital camera resolution measurement using sinusoidal Siemens stars,” Proc. SPIE, 6502 65020N (2007). https://doi.org/10.1117/12.703817 PSISDG 0277-786X Google Scholar

23. 

U. Artmann, “Quantify aliasing a new approach to make resolution measurement more robust,” Electron. Imaging, 2019 (10), 320-1 –320-6 (2019). https://doi.org/10.2352/ISSN.2470-1173.2019.10.IQSP-320 ELIMEX Google Scholar

24. 

P. G. Pitrone et al., “OpenSPIM: an open-access light-sheet microscopy platform,” Nat. Methods, 10 598 –599 (2013). https://doi.org/10.1038/nmeth.2507 1548-7091 Google Scholar

25. 

Y. A. Pan et al., “Zebrabow: multispectral cell labeling for cell tracing and lineage analysis in zebrafish,” Development, 140 (13), 2835 –2846 (2013). https://doi.org/10.1242/dev.094631 Google Scholar

26. 

C. Jaques et al., “Temporal resolution doubling in fluorescence light-sheet microscopy via a hue-encoded shutter and regularization,” OSA Continuum, 3 2195 –2209 (2020). Google Scholar

27. 

T. Blu and M. Unser, “Quantitative Fourier analysis of approximation techniques: part I—interpolators and projectors,” IEEE Trans. Signal Process., 47 2783 –2795 (1999). https://doi.org/10.1109/78.790659 ITPRED 1053-587X Google Scholar

28. 

T. Blu and M. Unser, “Quantitative Fourier analysis of approximation techniques: part II –wavelets,” IEEE Trans. Signal Process., 47 (10), 2796 –2806 (1999). https://doi.org/10.1109/78.790660 ITPRED 1053-587X Google Scholar

Biographies of the authors are not available.

© The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Christian Jaques and Michael Liebling "Aliasing mitigation in optical microscopy of dynamic biological samples by use of temporally modulated color illumination and a standard RGB camera," Journal of Biomedical Optics 25(10), 106505 (26 October 2020). https://doi.org/10.1117/1.JBO.25.10.106505
Received: 25 March 2020; Accepted: 24 September 2020; Published: 26 October 2020
JOURNAL ARTICLE
13 PAGES


SHARE
Advertisement
Advertisement
Back to Top