## 1.

## Introduction

The use of laser trapping devices, commonly known as optical tweezers, was first demonstrated by Ashkin in 1970.^{1} In the past two decades, the laser trapping technique has been further extended to realize multibeam optical tweezers and has been widely used for noncontact micro/nano-manipulation in various scientific fields, particularly in biomedical fields, for applications such as in Lab-on-a-Chip, bio-MEMS/NEMS, and microfluidic systems. The multibeam techniques used in optical tweezers such as holography,^{2} time-shared scanning,^{3} and generalized phase contrast (GPC),^{4} allow us to trap and manipulate many micro-objects simultaneously, in contrast to mechanical microhands^{5} which manipulate only one object at a time. As reported in our previous paper,^{6} we have developed a hybrid optical tweezers system, for dynamic handling of massive microbead arrays, consisting of two multibeam techniques including the GPC method using a spatial light modulator (SLM) and the time-shared scanning method using the galvano mirrors (GMs). This system was able to provide us with greater versatility, while the GPC method created the trap fields for immobilizing massive arrays, where the beads were manipulated smoothly and quickly by the GM scanning method. However, in the previous system, arrays formed by the GM scanning tweezers, based on the time-shared synchronized scanning (T3S) technique, could be handled only in a two and half dimensional (2.5D) working space. The manipulation in a 2.5D working space means that the controlled movement of objects in three-dimensional (3D) Cartesian coordinates is limited to 3D translations and two-dimensional (2D) rotations in $xy$-planes. This limitation arose from the lower bandwidth (several hertz in the previous system) of the $z$-axis manipulation because the manipulation was based on lens translation with a PC-controlled linear stage. The lens translation had high inertia and required millimeter order motion for the $z$-axis manipulation. Therefore, for the true 3D manipulation of micro-objects, an alternative $z$-axis manipulation method with higher bandwidth is required. In this paper, we present an improved hybrid optical tweezers system that can be used for the controlled 3D manipulation of multiple micro-objects with the help of an electrically focus-tunable lens with higher bandwidth.

## 2.

## Design Concept and Developed System

The 3D micro/nano-manipulation using optical tweezers is a significant technique for various scientific fields ranging from biology to nanotechnology. However, the use of optical tweezers alone is not sufficient to dexterously manipulate a nonspherical object in a true 3D working space or to automatically trap and transport objects for the following two reasons. First, to prevent biological objects from thermal damage, we generally use laser beams in the infrared (IR) region, therefore, we cannot exactly identify the actual focal positions of incident beams. Second, nonartificial objects generally have an inhomogeneous refractive index and a nonspherical shape, consequently, undesired torques, as well as forces, may be generated in the trapped objects. Hence, to achieve dexterous or automatic manipulation for various micro-objects, we have proposed the concept of a multibeam micromanipulation system which combines image processing and automatic control techniques.^{7}

Based on the above conceptual system design, we developed a hybrid optical tweezers system whose optical structure could be easily linked to an inverted microscope via its epi-fluorescence port. Figure 1 shows the optical and control system configurations of the developed system. The hybrid optical tweezers system uses two multibeam optical tweezers techniques which are the GPC method using an SLM and the T3S method using two commercially available fast-beam-steering devices. The single laser source is a continuous wave (cw) Nd:YAG laser (Laser Quantum, FORTE, 1064 nm, 700 mW, ${\mathrm{TEM}}_{00}$, England, United Kingdom); the beam emanating from the laser source passes through a half-wave plate ($\lambda /2$) and is split into two beams ($p$- and $s$- polarized beams) by a polarized beam splitter (PBS). One set of optical tweezers, based on the GPC method, is composed of an SLM, a phase contrast filter (PCF), and lenses (${\mathrm{L}}_{1}$ and ${\mathrm{L}}_{2}$) and it uses the $p$-polarized beam. The $s$-polarized beam is used to make the other set of optical tweezers which is based on the T3S method and is composed of an electrically focus-tunable lens ${\mathrm{L}}_{Z}$ (Optotune, EL-10-30-NIR-LD, $30\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mathrm{mm}\le {f}_{Z}\le \phantom{\rule{0ex}{0ex}}100\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mathrm{mm}$, response $\mathrm{time}=15\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mathrm{ms}$, Switzerland), a two-axis gimbal-mirror (Newport, FSM-300, California, USA), and two relay lenses (${\mathrm{L}}_{4}$ and ${\mathrm{L}}_{5}$). The closed-loop amplitude bandwidth of the gimbal-mirror is 800 Hz and the resolution is 1 *μ*rad [$\pm 26.2\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mu \mathrm{rad}$ for $\pm 0.01\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mathrm{V}$ of digital-analog (DA) voltages]. The laser power can be distributed between the two methods in varying proportions with the half-wave plate. The optical potential energy landscapes^{8} created by these tweezers can be controlled independently since the $p$- and $s$-polarized beams do not interfere with each other.

In this new hybrid system, the GPC part is the same as that in our previous hybrid system,^{6} the T3S part is redesigned for true 3D-T3S manipulation based on the design guidelines for stable 3D scanning traps, reported in our previous paper,^{9} while retaining the design concept and system features of the previous hybrid system. Consequently, the T3S part enables us to independently and simultaneously control several trap positions in a true 3D working space while the GPC part provides massive 2D traps (that is, over several tens of traps). The designed steering ranges are $-78.6\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mu \text{m}\le \delta xy\le 78.6\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mu \text{m}$ and $-74.3\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mu \text{m}\le \delta z\le 67.4\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mu \text{m}$ for the $\times 60$ objective (Olympus, UPlanFLN $\times 60$, ${f}_{o}=3.0\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mathrm{mm}$, Japan), ${f}_{4}=\phantom{\rule{0ex}{0ex}}100\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mathrm{mm}$, and ${f}_{5}=200\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mathrm{mm}$. The number of the beads that can be handled by the T3S part is limited to $16/8$ for a 2D/3D manipulation owing to the dwell time of scanning (typically, 10 or 15 ms for 2 *μ*m beads), which depends on the bead size and the response time of both the gimbal-mirror and the focus tunable lens. The advantage of our system is the direct control of 2D/3D trap positions using both the bitmap images and the DA voltages which have a one-to-one correspondence with the 2D/3D trap positions. Therefore, compared to other systems based on the holographic method, we can manipulate multiple objects, in a true 3D working space, smoothly and precisely without the presence of ghost traps^{10} in the holographic tweezers.

## 3.

## Demonstrations of 3D Manipulation

## 3.1.

### Controlled Manipulation along 3D Paths

Here, using the T3S part alone, we demonstrate that the controlled manipulation of five beads along 3D paths, defined by explicit functions of time $t$, can be simply achieved in real time without prior computation of their manipulation paths. Figure 2 (Video 1) shows snapshots captured with a CCD camera and presents the results of the controlled manipulation of five beads (Duke Scientific, borosilicate microsphere, $2.5\pm 0.5\text{\hspace{0.17em}}\text{\hspace{0.17em}}\mu \text{m}$, California, USA). The laser power at the entrance pupil of the objective lens was 155 mW and the scanning dwell time for each bead was 15 ms. Based on the time-shared scanning approach, the $x$-, $y$-, and $z$-coordinates of each bead, at time $t$, were controlled according to the functions

## (1)

$$\{\begin{array}{l}{{}^{\theta}x}_{k}(t,{\theta}_{x})={x}_{k}(t),\\ {{}^{\theta}y}_{k}(t,{\theta}_{x})={y}_{k}(t)\mathrm{cos}\text{\hspace{0.17em}}{\theta}_{x}-{z}_{k}(t)\mathrm{sin}\text{\hspace{0.17em}}{\theta}_{x},\\ {{}^{\theta}z}_{k}(t,{\theta}_{x})={y}_{k}(t)\mathrm{sin}\text{\hspace{0.17em}}{\theta}_{x}+{z}_{k}(t)\mathrm{cos}\text{\hspace{0.17em}}{\theta}_{x},\end{array}$$## (2)

$$\{\begin{array}{l}{x}_{k}(t)=A\text{\hspace{0.17em}}\mathrm{cos}(2\pi t+\frac{2\pi}{5}k),\\ {y}_{k}(t)=A\text{\hspace{0.17em}}\mathrm{sin}(2\pi t+\frac{2\pi}{5}k),\\ {z}_{k}(t)=\mathrm{const},k=1,\dots ,5.\end{array}$$Consequently, the five trapped beads, which formed the shape of a pentagon owing to the 3D time-shared scanning, were interactively rotated by an angle of ${\theta}_{x}$ about the $x$-axis in Cartesian coordinates while the pentagon was rotated about its center.

## 3.2.

### Controlled Rotation of Four Beads about Arbitrary Axes in 3D Space

Here, using the T3S part alone, we also demonstrate that the controlled rotation of four beads, which hypothetically form a single rigid body (that is, a tetrahedron), about an arbitrary axis in Cartesian coordinates, can be simply performed using “homogeneous transformation” in computer graphics,^{11} in real time. Figure 3 (Video 2) shows snapshots captured with a CCD camera and presents the results of the interactive and controlled 3D rotation of four unconnected beads that form a tetrahedron. The homogeneous representation of the tetrahedron in Fig. 3(a) is

## (3)

$$\mathbf{P}=\left[\begin{array}{cccc}-1& 1& 0& 0\\ -\frac{\sqrt{3}}{3}& -\frac{\sqrt{3}}{3}& \frac{2\sqrt{3}}{3}& 0\\ 0& 0& 0& 1\\ 1& 1& 1& 1\end{array}\right],$$## (4)

$${\mathbf{A}}_{11x}=\left[\begin{array}{ccc}1& 0& 0\\ 0& \mathrm{cos}\text{\hspace{0.17em}}{\theta}_{x}& -\mathrm{sin}\text{\hspace{0.17em}}{\theta}_{x}\\ 0& \mathrm{sin}\text{\hspace{0.17em}}{\theta}_{x}& \mathrm{cos}\text{\hspace{0.17em}}{\theta}_{x}\end{array}\right]\phantom{\rule[-0.0ex]{1em}{0.0ex}}\mathrm{and}\phantom{\rule[-0.0ex]{1em}{0.0ex}}{\mathbf{A}}_{12}=\left[\begin{array}{c}0\\ 0\\ 0\end{array}\right],$$## (5)

$${\mathbf{A}}_{11y}=\left[\begin{array}{ccc}\mathrm{cos}\text{\hspace{0.17em}}{\theta}_{y}& 0& \mathrm{sin}\text{\hspace{0.17em}}{\theta}_{y}\\ 0& 1& 0\\ -\mathrm{sin}\text{\hspace{0.17em}}{\theta}_{y}& 0& \mathrm{cos}\text{\hspace{0.17em}}{\theta}_{y}\end{array}\right]\phantom{\rule[-0.0ex]{1em}{0.0ex}}\mathrm{and}\phantom{\rule[-0.0ex]{1em}{0.0ex}}\phantom{\rule{0ex}{0ex}}{\mathbf{A}}_{11z}=\left[\begin{array}{ccc}\mathrm{cos}\text{\hspace{0.17em}}{\theta}_{z}& -\mathrm{sin}\text{\hspace{0.17em}}{\theta}_{z}& 0\\ \mathrm{sin}\text{\hspace{0.17em}}{\theta}_{z}& \mathrm{cos}\text{\hspace{0.17em}}{\theta}_{z}& 0\\ 0& 0& 1\end{array}\right],$$## 3.3.

### Automated Assembly and Interactive 3D Manipulation of Arrays

By using both the GPC tweezers and the T3S tweezers, we have demonstrated the automated assembly of microbead arrays and the subsequent 3D manipulation of the arrays. Figure 4 (Video 3) shows a sequence of images recorded with a CCD camera and shows the results of the fully automated assembly and immobilization of two squares made from microbeads using only the GPC part and the subsequent translation of a single array (that is, the inner $2\times 2$ array) in a 3D working space using the T3S technique. The laser power for the GPC tweezers was 156 mW and that for the T3S tweezers was 50 mW. The sample is polystyrene microbeads (Polysciences, 2 *μ*m, Pennsylvania, USA). First, in order to fully and automatically assemble the arrays of microbeads (that is, the two square arrangements of microbeads), the center positions of all beads in an image were detected by the Hough transform^{12} and the subsequent irradiation of disk-shaped beams based on the GPC method trapped the 28 beads at their initial detected positions, where all beads were trapped against the lower surface of an upper cover glass in the same $xy$-plane, namely, the microscope’s imaging plane [Fig. 4(a)]. Second, under a modified version of the control algorithm described in our previous paper,^{13} the 28 trapped beads were transported using only the GPC tweezers to form two squares [Fig. 4(b)–4(d)]. Third, four beads forming the inner $2\times 2$ array were firmly and simultaneously trapped by the T3S tweezers instead of the GPC tweezers, while the 24 beads forming the outer square remained trapped by the GPC tweezers. Finally, subsequent 3D translations of the $2\times 2$ array, namely, lowering of the smaller square [Fig. 4(e)], lateral translation of that square to outside the large square [Fig. 4(f) and 4(g)], and raising of the smaller square back to the original level [Fig. 4(h)], could successfully transport the square in 3D space. Consequently, the $2\times 2$ array, which was initially inside the large square, was able to reach the outside of the large square while maintaining its geometrical shape. Figure 4(i) illustrates these 3D translations of the $2\times 2$ array in a cross-sectional view where yellow circles indicate the $2\times 2$ array and red circles indicate the large square formed by the 24 beads.

## 3.4.

### Discussions

In the two demonstrations using the T3S part alone, the initial loadings of beads to the 3D arrays (that is, a pentagon and a tetrahedron) were autonomously completed by moving the microscope’s XY-table. Because their 3D geometry is simple and the strongly focused, scanning traps based on geometrical optics can be exactly generated in 3D Cartesian coordinates without ghost traps that exist in the holographic methods. In order to assemble complex 3D geometries, in the future work, the initial loading process will be automated by developing both collision-free control algorithms^{13} for 3D arrays and 3D position monitoring techniques.^{14}^{,}^{15}

On the other hand, in the third demonstration using both the GPC tweezers and the T3S tweezers, the initial loading of beads was fully automated using the Hough transform because all detected beads at their initial positions were able to suspend in the single $xy$-plane (that is, 2D working space) by the GPC beams. The subsequent process of our third demonstration was performed, in the strict sense of the word, in a 2.5D working space. However, in the future work, the application of the morphing process^{16} from a 2D array to a 3D array will be able to fully automatically assemble 3D structures only using a collision-free control algorithm. In the morphing process using the 3D-T3S tweezers, no 3D position monitoring of trapped beads may be required after amplitude calibration because such morphing of 3D traps can be exactly controlled by DA voltages that have a one-to-one correspondence with the 3D trap positions.

## 4.

## Conclusion

For automated true 3D manipulation of multiple micro-objects, we developed an improved hybrid optical tweezers system consisting of GPC tweezers with an SLM and 3D-T3S tweezers with a two-axis fast-steering mirror and an electrically focus-tunable lens and we demonstrated three examples of controlled 3D manipulation. Although the demonstrations performed are simple, the developed system can be used as a valuable noncontact manipulation tool in various scientific fields ranging from biology to nanotechnology. Additionally, the presented hybrid system offers an alternative platform for the optical assembly^{17} and probing^{18} of 3D microstructures because our system, which can be easily linked to a standard microscope, can in a simple, precise, and independent manner, control 3D multiple traps under visual control schemes by machine vision using the bitmap images/DA voltages that have a one-to-one correspondence with the 2D/3D trap positions.

## References

## Biography

**Yoshio Tanaka** is a senior research scientist of National Institute of Advanced Industrial Science and Technology (AIST), AIST Shikoku in Japan. He received the BE, ME, and PhD degrees in electronic engineering from the University of Tokushima in 1982, 1984, and 1997, respectively. In 1986, he joined Government Industrial Research Institute, Shikoku, MITI of Japan, the predecessor of the present AIST Shikoku, where he was engaged in research on robotics and mechatronics. Now, his research interests include opto-mechatronics, bio-MEMS/NEMS, and micromanipulation using laser trapping for various scientific fields.

**Shogo Tsutsui** received the BE and ME degrees in reliability-based information systems engineering from Kagawa University in 2009 and 2011, respectively. He has been working on development of a hybrid optical tweezers system combined with intelligent control techniques.

**Hiroyuki Kitajima** is an associate professor of Graduate School of Engineering, Kagawa University in Japan. He received the BE, ME, and PhD degrees in electrical and electronic engineering from the University of Tokushima in 1993, 1995, and 1998, respectively. In 1999, he joined Kagawa University. Now, his research interests include bifurcations and synchronizations in nonlinear dynamical systems.