Infrared up-conversion imaging in nonlinear metasurfaces

Infrared imaging is a crucial technique in a multitude of applications, including night vision, autonomous vehicles navigation, optical tomography, and food quality control. Conventional infrared imaging technologies, however, require the use of materials like narrow-band gap semiconductors which are sensitive to thermal noise and often require cryogenic cooling. Here, we demonstrate a compact all-optical alternative to perform infrared imaging in a metasurface composed of GaAs semiconductor nanoantennas, using a nonlinear wave-mixing process. We experimentally show the up-conversion of short-wave infrared wavelengths via the coherent parametric process of sum-frequency generation. In this process, an infrared image of a target is mixed inside the metasurface with a strong pump beam, translating the image from infrared to the visible in a nanoscale ultra-thin imaging device. Our results open up new opportunities for the development of compact infrared imaging devices with applications in infrared vision and life sciences.


Introduction
Infrared (IR) spectroscopy and imaging are growing in demand due to the increasing number of applications in this spectral region, including optical tomography, 1 process monitoring, 2 food and agriculture quality control, 3 night vision devices, 4 as well as LIDAR and remote sensing. 5,6 Commercial IR imaging detectors rely on the absorption of incident photons in narrow band gap materials and the release of electrons that are electrically detected. However, due to the low IR photon energy, such IR detection schemes require low-temperature and even cryogenic cooling. As a result, IR cameras are generally bulky, containing several components for photon-electron conversion.
An alternative scheme, which can potentially overcome the limitations of photoconductive detectors, is the use of nonlinear optical processes for up-conversion of the energy of photons.
In this approach, the IR image is not detected directly, instead a parametric nonlinear optical process is employed to convert the image to higher frequencies and detect it using regular cameras, in a process known as up-conversion IR imaging. In 1968, Midwinter first demonstrated IR up-conversion imaging by converting the IR signal to the visible spectrum using a nonlinear crystal and the aid of a pump beam. 7 In his work, the spatial information of a short-wave IR (SWIR) image (λ = 1.6 µm) was coherently transferred to the visible domain (λ = 0.484 µm), using a parametric second-order nonlinear process known as sumfrequency generation (SFG). In the SFG process, two incident waves with frequencies ω 1 and ω 2 interact inside a second-order nonlinear material leading to sum-frequency emission with frequency ω SF G = ω 1 + ω 2 , as shown in Figure 1a. The results obtained by Midwinter showed the possibility of detecting IR images with relatively high sensitivities using standard, fast and uncooled Si-based detectors.
A decade of intensive research followed the first demonstration of up-conversion IR imaging, where the performance and resolution of the imaging systems were studied in different arrangements, including various incidence angles, 8 nonlinear bulk crystals, 9 optical configurations, 10 bandwidth 11 of the IR radiation, and pump beams. 12 However, the low signal to noise ratio of CCD detectors and poor quality of nonlinear crystals at the time prevented practical developments of up-conversion imaging systems. Recently, the interest in such imaging systems has been renewed, driven by the availability of periodically-poled nonlinear crystals, [13][14][15] new laser sources [16][17][18] and the use of intra-cavity configurations, 14,19 which can improve the performance of the systems. The main difficulty in the realization of these IR imaging systems is the phase-matching condition, which not only restricts the conversion efficiency of the up-conversion process but also limits the spectral bandwidth of the IR image, resolution and field of view.
Here, we propose a novel approach to perform IR up-conversion imaging using for the first time, to the best of our knowledge, nanostructured ultra-thin metasurfaces. Our metasurfaces, composed of fabricated nanoantennas on (110) GaAs wafers, are resonant at all the interacting wavelengths. Thus by employing SFG within the resonant metasurface, we demonstrate nonlinear wave-mixing of a SWIR signal beam with a near-IR pump beam, to generate an up-converted emission in the visible spectrum. More importantly, when the IR signal beam carries the image of a target, the spatial information of the target is preserved in the nonlinear wave-mixing process despite being generated by hundreds of independent GaAs crystalline nanoantennas. Therefore, the ultra-fast nonlinear up-conversion process enables IR imaging with femtosecond temporal resolution. Such advancement opens up future opportunities for ultra-fast imaging of chemical reactions in a conventional microscope device.

Nonlinear metasurfaces for IR imaging
Metasurfaces are planar arrays of densely packed nanoantennas designed to manipulate the properties of incident light, including its amplitude, directionality, phase, polarization and frequency. 20 The optical response of metasurfaces is governed by the collective scattering of individual nanoantennas and the mutual coupling among neighboring nanoantennas. Recent advances in nanofabrication technologies 21 have motivated extensive research in the field of metasurfaces. Among various examples, dielectric and semiconductor metasurfaces have shown great promise for enhancing nonlinear optical processes at the nanoscale. 22 Such metasurfaces can exhibit enhanced frequency conversion due to the excitation of optical resonances [23][24][25][26] and good coupling to free-space.
However, the strongest nonlinear response of materials originates from quadratic nonlinearity, which is present only in non-centrosymmetric materials. GaAs and its aluminum alloys are often the materials of choice for quadratic nonlinear metasurfaces, being III-V semiconductor materials that possess a zinc-blende non-centrosymmetric crystalline structure, and high quadratic nonlinear susceptibility χ (2) ∼ 200 pm/V. 27 Nevertheless, the use of GaAs compounds comes with significant challenges due to off-diagonal symmetry of its second-order nonlinear susceptibility tensor. While (100)-GaAs metasurfaces have been used to demonstrate ultra-thin second harmonic sources, 28-30 frequency-mixers 31 and directional lasing, 32 harmonic emission from such metasurfaces is forbidden at normal incidence. 30,33 Recent studies have aimed at directing the harmonic emission at normal direction to the metasurface, 33-36 however, the most successful strategy to date has been to the use of different crystalline symmetry nanoantennas. Indeed, the GaAs χ (2) tensor is not invariant under rotation of the crystallographic axes, thus for (111) and (110) GaAs metasurfaces, the diagonal components of the nonlinear susceptibility tensor, χ (2) rot are different from zero. Normal second harmonic generation (SHG) was first demonstrated in (111) AlGaAs nanoantennas. 37 However, (110) GaAs nanoantennas have shown highly directional SHG and unique control of its forward to backward emission. 38 Such highly directional normal emission promotes the nonlinear mixing of two co-propagating beams to generate sum-frequency emission also propagating along the normal direction. Therefore, in our work we employ (110) GaAs metasurfaces to perform IR up-conversion imaging through the SFG process. In this way, our metasurface can mimic a bulk nonlinear crystal and perform co-linear wave-mixing without the need of co-linear phase-matching.
The process of IR up-conversion imaging in a nonlinear metasurface is schematically represented in Figure 1b. In this Figure, the image of a target (Siemens star) is encoded in the IR signal beam (red beam) and up-converted to a visible image (green beam) due to the nonlinear wave-mixing of signal and pump beams (orange beam) within the metasurface. In the rest of the paper, the colors red, orange and green in the figures will be used to refer to the signal, pump and SFG beams, respectively. In our configuration, the pump beam and the IR image of a target in the signal beam, are simultaneously focused on the metasurface (see left-hand side of Figure 1b). The pump and signal beams are mixed together within the GaAs metasurface through the SFG process (see energy diagram in Figure 1a

Numerical results
First, we designed the linear optical properties of (110) GaAs metasurfaces to support resonances at the wavelengths of the signal and pump beams. Different geometric parameters of the metasurface were optimized to obtain the desired resonances, namely the disk nanoantenna radius, r and the array periodicity, P (see Figure 2c). In our calculations, the height of the nanoantennas, h is fixed to 400 nm. The 2D transmission maps obtained by varying the nanoantennas separation from 600 to 1000 nm, and the nanoantennas radius from 175 to tained. The forward SFG conversion efficiency, η = P SF G /P s of the designed metasurface is 1.6 × 10 −6 for I p = 0.78 GW/cm 2 and I s = 0.38 GW/cm 2 , corresponding to the typical values in our measurements. This efficiency is dependent on the pump power, therefore the normalized conversion efficiency η N orm = η/P p , is a better measure of the efficiency of the SFG process. Here P p is the average power of the pump beam.
According to the final design of the metasurface, the spatial field profiles in a metasurface unit cell were calculated at the pump and signal wavelengths. These field profiles normalized to the incident electric field are shown in Figure 2d and e, respectively. In each case, the electric field profile is shown in the middle xy-plane of the nanoantenna (see Figure 2c).
The spatial field profile of the pump ( Figure 2d) shows a maximum enhancement of about

Nonlinear emissions from GaAs metasurfaces
Next, we measured the SFG intensity by independently tuning the wavelengths of the signal and pump beams around the spectral region of interest. We use a Ti:Sapphire laser with an optical parametric oscillator (OPO) which together deliver two pulsed train beams (see Supporting Information Figs. S5 and S7) with a repetition rate of 80 MHz. First, the wavelength of the signal beam was fixed at 1530 nm and the wavelength of the pump was tuned from 830 to 880 nm (see Figure 4a). Both beams were linearly polarized along the horizontal direction. The spectrum in Figure 4a shows the emission of the SFG from 537 to 558 nm, with a maximum efficiency at 549 nm, corresponding to an excitation pump beam at 860 nm. After exhibiting a maximum at 549 nm, the SFG intensity decreases with the increase of the pump wavelength. The use of a pump beam with a wavelength longer than 880 nm is limited by our laser system.
Next, the wavelength of the signal beam was tuned from 1470 to 1570 nm, while maintaining the pump beam fixed at 860 nm (see Figure 4b). The wavelength of the pump beam was chosen according to the maximum SFG observed in Figure 4a. Figure 4b shows the SFG emission from 541 to 555 nm, with a maximum at 549 nm, corresponding to a signal beam at 1530 nm. After the maximum at 549 nm, the SFG intensity gradually decreases with the increase of signal wavelength. Through all these measurements, the average power of the pump and signal beams, measured right before the metasurface, was kept constant at 18 and 14 mW, respectively. As can be seen in Figures 4a and b, the optimized SFG intensity takes place when the metasurface is excited by a pump beam at 860 nm and a signal beam at 1530 nm. This behavior is explained by the near-field enhancement of the excitation beams, when the metasurface is resonantly excited (see Figure 3c and d) at these excitation wavelengths.
The visible nonlinear spectrum of the metasurface is shown in Figure 4c, characterized by two strong nonlinear emissions at 430 and 550 nm. The metasurface was excited by signal and pump beams at the optimized wavelengths using an average power of 10 mW in each beam, measured before the metasurface. The shorter wavelength nonlinear emission at 430 nm originates from the SHG of the pump (2ω p ), while the emission at 550 nm originates from the SFG process (ω s + ω p ). Other nonlinear processes are also generated in the metasurface at wavelengths longer than the SFG, however, these wavelengths are blocked by the shortpass filter (with cut-off wavelength of 600 nm) used to filter out the transmitted pump beam (see Supporting Information, Fig. S5), thus they are not collected by our detection system.
Above the band gap of the GaAs (1.42 eV, 873 nm), the absorption coefficient increases as the incident wavelength decreases. Therefore, in Figure 4c the higher intensity of the SHG 2ωp , as compared to the SFG intensity, is unexpected. Traditionally, the stronger nonlinear intensity of a metasurface is associated to the near-field enhancement at the fundamental wave. 26,31 However, the efficiency of the nonlinear frequency-mixing also depends on the spatial mode overlap of the interacting waves [40][41][42] (Figure 2d and e). Inside the GaAs nanoantennnas the field enhancement of signal and pump beams has similar intensity (see Supporting Information, Fig. S2, as this is not reflected in the particular cross-section of Figures 2d and e). Therefore, the higher intensity of the SHG (ω p + ω p ) can be attributed to the full spatial overlap of the pump field with itself ( Figure 2c). Whereas, in the case of SFG (ω s + ω p ) the spatial overlap between the signal and pump beams (Figures 2c and d) is not complete, thereby the intensity of the SFG emission is weaker. It is worth noting that the SFG intensity is also dependent on the spatial overlap between the excitation and SFG fields (see Supporting Information, Fig. S3). Further studies can be performed to investigate the relative intensities of the nonlinear emissions. However, these studies are outside the scope of our work.
To verify the origin of the nonlinear emissions generated by the metasurface, the average power of the pump beam was gradually increased from 2 to 20 mW with a 2 mW step, while keeping the power of the signal beam constant. The intensity of the parametric emissions was recorded and analyzed on a log-log plot, as shown in Figure 4d. In the case of SHG 2ωp a quadratic dependence on the pump power was obtained with a slope of 2.11; whereas As an important consequence, when short pulses are used for both signal and pump beams in the SFG process, these two pulses require temporal synchronization (∆t = 0), as illustrated in the top of Figure 5a. In contrast, the SHG 2ωp is a degenerated nonlinear process where two photons come from the same pulse, thus no temporal synchronization is required in this case (see bottom of Figure 5a). Here, we achieved temporal synchronization of the signal and pump pulsed beams by using a free-space variable delay line with micrometer adjustment to finely adjust the path length of the pump beam (see Supporting Information, Fig. S5), and thus to accurately control the time delay between pulses. The spectra of the up-converted nonlinear emissions were measured as a function of the time delay, as illustrated in Figure 5b. The experimental conditions used in these measurements were the same as the ones used in Figure 4c. It is worth noting that the SHG 2ωp is independent of time delay, while the strongest SFG emission is generated at zero time delay. As can be seen in Figure 5b, when the time delay changes from zero to ±333 fs, the SFG intensity continuously drops until becoming negligible. The temporal duration of the SFG emission was measured to be 267 fs, which is effectively the convolution of the signal and pump pulses (see Supporting Information, Fig. S7). This finding demonstrates that the up-conversion process preserves the temporal information of the femtosecond IR pulses and can find applications in ultra-fast IR imaging of dynamic phenomena. 43

Infrared imaging
Finally, we present the up-conversion IR imaging enabled by the GaAs metasurface. In this experiment, the collimated signal beam passes through a Siemens star target (see Supporting Information, Fig. S6), which is imaged by the focusing lens onto the metasurface. The IR image in Figure 6a (second frame) was acquired with an InGaAs IR camera (Xenics, XS-1.7-320), using only signal beam illumination. The section of the Siemens star imaged is highlighted by a red square. At the metasurface, this SWIR image is mixed with the mildly focused pump beam and through the SFG process converted to a visible wavelength ( Figure 6a, last frame). Importantly, the visible images presented here have been captured using a conventional CCD camera (Starlight Xpress, SXVR-H9). Ideally, since the SFG is When the pump and signal beam are temporally detuned, the visible images shown in Figure 6b completely vanished, thus corroborating they are only formed by the SFG process.
In our experiment, the resolution of the up-converted images is limited mainly by the SWIR imaging of the target, as seen from the IR image in Figure 6a. Ultimately, the fundamental limit on the resolution of the visible images is the periodicity and size of the individual nanoantennas. In our case this fundamental resolution is of the order of 750 nm. Overall, the GaAs metasurface enables high-contrast and low-noise IR imaging at room temperature, which are great advantages when compared to other competing technologies.

Conclusions
In conclusion, we have demonstrated for the first time, to the best of our knowledge, upconversion of an IR image to visible wavelengths by a resonant ultra-thin GaAs metasurface. The up-conversion is realized by nonlinear wave-mixing of SWIR images with a strong near-IR pump beam within the metasurface. The ultra-fast nonlinear conversion of the sum-frequency process is dramatically enhanced in our 400 nm-thick metasurface due to the excitation of optical resonances at all the three interacting waves. In this way, the IR signal can be easily detected with a simple-uncooled CMOS camera. The realized up-conversion process is parametric and does not exchange energy with the environment, and as such, all spatial information encoded into the IR signal beam is preserved during the up-conversion.
Despite different parts of the IR signal beam being up-converted by independent nanoantennas composing the metasurface, the images are well reproduced into the visible, with the ultimate resolution limit being the periodicity of the metasurface.
Unlike current IR cameras, our all-optical approach is not affected by thermal noise and can operate at room temperature using conventional CMOS detectors. Importantly, our metasurface-based IR imaging approach offers novel opportunities, not possible in conventional up-conversion systems where bulky nonlinear crystals are used. For example, the nonlinear wave-mixing can be obtain for counter-propagating pump and signal beams, as well as for incidence at all different angles, as long as the metasurface resonances are excited.
Most importantly, multi-color SWIR imaging is also possible by an appropriately designed metasurface. In that case, the designed metasurface would be composed of nanoantennas with different sizes, having resonances at different IR signal wavelengths, while maintaining fixed the resonance of the pump beam. Such metasurface would be able to convert several IR wavelengths to the visible, according to energy conservation observed by the SFG parametric nonlinear process (ω SFG = ω s + ω p ).
We note that our SFG conversion efficiency can be further optimized and enhanced using several strategies, including the use of high-quality factor resonances 24,44 and materials with higher transparency in the visible region. 45 Additional optimization could also be achieved by employing machine learning approaches for enhancing light-matter interactions. 46 We believe that by enhancing the SFG conversion efficiency, continuous-wave nonlinear up-conversion is within reach.
Our results can directly benefit the development of compact night vision instruments and sensor devices. Notably, the demonstrated SWIR metasurface imaging devices can be ultra-thin and ultra-compact, be fabricated on flexible substrates, and be fully transparent. In addition, they could offer new functionalities such as multi-color imaging at room-temperature. Experimental setup. The optical system used for the nonlinear characterization of the GaAs metasurface is described below. First, the output of a tunable mode-locked Ti:Sapphire laser is split into two beams. One of the beams is coupled to an Optical Parametric Oscillator (OPO) to obtain an IR signal beam, while the other beam is directly used as the pump beam. The tuning range of the pump beam is 740 to 880 nm, while the tuning range of the signal beam is 1000 to 1600 nm. The pump beam passes through an optical delay line (see Supporting Information, Fig. S5), while the signal beam encodes the image of a target through an imaging system ( Supporting Information, Fig. S6). The temporal duration of the pump and signal pulses is measured using a frequency-resolved optical grating method (see Supporting Information, Fig. S7a.). The excitation beams are then spatially combined by a dichroic mirror and focused by a lens on the metasurface. The nonlinear emissions generated by the metasurface are collected by an objective lens and sent to a CCD camera or an spectrometer. For more details see Supporting Information.

Supporting Information Numerical calculations
The linear and nonlinear optical response of the (110) GaAs metasurface is numerically modeled by using the Finite Element Method in Comsol Multiphysics. The nonlinear response is obtained in a two-step approach. First, we calculate the nonlinear polarization response of the metasurface P(ω 3 ) resulting from the incident pump and signal frequency beams. Then, we employ the nonlinear polarization as the source to calculate the SFG, through the induced nonlinear current J(ω 3 ). We define the i -th component of the nonlinear electric polarization vector at the angular frequency ω 3 as with i = j = i due to the zinc-blende crystal structure of GaAs. Here 0 is the vacuum permittivity, E j (ω 1 ) is the j -th component of the electric field at the angular frequency ω 1 and E k (ω 2 ) is the k -th component of the electric field at the angular frequency ω 2 . The angular frequencies ω 1 , ω 2 and ω 3 correspond to the wavelength of the signal at 1530 nm, the pump at 860 nm, and the SFG at 550 nm, respectively. The GaAs metasurface is simulated by implementing Floquet boundary conditions to mimic an infinite 2D periodic structure. Figure S1 shows the calculated reflection of our GaAs metasurface (P = 750 nm, r + 225 nm and h = 400 nm), as a function of incident wavelength. As shown in Figure S1, at long wavelengths the reflection spectrum has mainly electric and magnetic dipole contribu-   When considering the lattice effects of the GaAs metasurface, the sum-frequency emission will be shaped into different diffraction orders, depending on the periodicity P of the metasurface. Figure S4 shows the sum-frequency diffraction coefficients of our GaAs metasurface generated by the simultaneous incidence of an IR signal beam at 1530 nm and a pump at 860 nm. The SFG diffraction coefficients are calculated by performing the Fourier transform of the sum-frequency near field in both directions, backward and forward. As indicated by the color intensity scale in Figure S4a and b, in each direction the strongest SFG emission corresponds to the zero-th diffraction order. In both directions, the first diffraction orders in the x -and y-directions have lower intensity than the zero-th diffraction order. Overall, the forward SFG intensity is stronger than the backward SFG. In our metasurface, there are no second diffraction orders for the sum-frequency emission.

Experimental setup and measurements
The schematic of the optical system used to characterize the nonlinear optical response of the GaAs metasurface is shown in Figure S5.  The measurements were performed at the output of the OPO cavity (see Figure S5), before the half-wave plates. Due to the femtosecond duration of the pulses, the dispersion effects introduced in the excitation beams when they travel through glass (focusing lens, half-wave plates, etc) are considered negligible. Thus, the duration of the pulses measured at the output of the OPO cavity correspond to the duration of the pulses at the metasurface plane.  Figure S5: Schematic of optical system used to study the nonlinear emissions generated by (110) GaAs metasurface. The schematic shows the optical path of the signal and pump pulsed laser beams employed to excite the metasurface and generate sum-frequency emission at green wavelengths. The IR up-conversion imaging is performed by thusing the imaging system in the IR signal beam. In the schematic, the focused pump and signal beams are not spatially overlapped only for visualization purposes.
Imaging system NA 0.14 L f WD Target Figure S6: Schematic of imaging system employed to encode real image of a Siemens star target in the IR signal beam (and subsequently in the SFG beam). The system consists of a lens (L) with f = 100 mm and an objective lens with NA of 0.14, placed in a confocal configuration. Figure S7: Cross correlation of the pump pulsed laser beam with the signal pulsed laser beam. (a) The duration of the signal and pump laser beams was directly measured, giving a pulse-width of 155 and 168 fs, respectively. (b) The duration of the SFG emission was measured using an optical delay line, giving a pulse-width of 267 fs.