Translator Disclaimer
28 August 2020 Laparoscopic system for simultaneous high-resolution video and rapid hyperspectral imaging in the visible and near-infrared spectral range
Author Affiliations +
Abstract

Significance: Hyperspectral imaging (HSI) can support intraoperative perfusion assessment, the identification of tissue structures, and the detection of cancerous lesions. The practical use of HSI for minimal-invasive surgery is currently limited, for example, due to long acquisition times, missing video, or large set-ups.

Aim: An HSI laparoscope is described and evaluated to address the requirements for clinical use and high-resolution spectral imaging.

Approach: Reflectance measurements with reference objects and resected human tissue from 500 to 1000 nm are performed to show the consistency with an approved medical HSI device for open surgery. Varying object distances are investigated, and the signal-to-noise ratio (SNR) is determined for different light sources.

Results: The handheld design enables real-time processing and visualization of HSI data during acquisition within 4.6 s. A color video is provided simultaneously and can be augmented with spectral information from push-broom imaging. The reflectance data from the HSI system for open surgery at 50 cm and the HSI laparoscope are consistent for object distances up to 10 cm. A standard rigid laparoscope in combination with a customized LED light source resulted in a mean SNR of 30 to 43 dB (500 to 950 nm).

Conclusions: Compact and rapid HSI with a high spatial- and spectral-resolution is feasible in clinical practice. Our work may support future studies on minimally invasive HSI to reduce intra- and postoperative complications.

1.

Introduction

Hyperspectral imaging (HSI) has been used in many fields of biomedical research and clinical applications in recent years.1 Promising results were reported for the estimation of organ perfusion,2,3 differentiation of anatomical structures,4,5 wound assessment,6,7 and the detection of cancerous tissue811 in animal experiments and clinical studies.12 However, the majority of the described systems do not apply to current minimally invasive surgery. Supporting the estimation of tissue perfusion or detection of risk structures with spectral imaging would be particularly important in laparoscopic procedures where surgeons are limited to their visual impression.

Reported multi- and hyperspectral endoscopic systems since 2013 are listed in Table 1. Two systems are capable of investigating both the visible and near-infrared range, but are limited by long acquisition times and without color video during spectral imaging or low spectral resolution.13,14 Simultaneous hyperspectral data and color video acquisition in the visible range were achieved with pushbroom and manual line-scanning flexible endoscopes.15,16 Most systems are using spectral scanning methods like acousto-optical tunable filter (AOTF), liquid crystal tunable filter (LCTF), filter wheels, or light sources with monochromators. Spectral scanning typically results in high spatial resolution but longer acquisition times per spectral channel and misalignments due to motion. Spectrally resolving detector arrays (SRDA) or snap-shot imagers are nonscanning systems and are, therefore, not limited by motion artifacts.13 However, spatial and spectral resolutions are reduced. Spatial or pushbroom scanning allows high spectral and spatial resolution at the same time. Previously described endoscopic pushbroom systems have the disadvantage of having to move the sample or the endoscope itself.14,16

Table 1

Specifications of reported multi- and hyperspectral laparoscopic (HSI/MSI) systems since 2013.

AuthorSystem description [light source]Spectral separationSpectral range (nm)Spectral channels [resolution in nm]Image size in pixels [spatial resolution]Acquisition [processing] time in sColor videoaMedical application
Clancy et al.21MSI laparoscope [xenon]LCTF500 to 62013 [7 to 20]512×384 [0.7 mm]7 [45]NoPerfusion
Fawzy et al.22MSI flexible endoscope [xenon]Filter wheel400 to 76018 [15]200×200 [n.a.]0.07 [2.4]NoPerfusion
Hohmann et al.23MSI flexible endoscope [xenon]n.a.400 to 6506 [12 to 20]350×370 [50 to 500  μm]0.45 [1.5]NoLG CA
Luthman et al.13MSI flexible endoscope [LED]2 × SRDA470 to 63016 [9 to 26]n.a. [300 to 400  μm]0.01 [600]YesU/LG CA
600 to 100025 [7 to 15]
More et al.24MSI rigid endoscope [halogen]Line scanning monochromator480 to 70516 [15]1392×1024 [n.a.]20 [n.a.]NoMice retina
Zhang et al. 25MSI laparoscope [n.a.]Filter wheel470 to 7008 [20 to 25]300×300 [6.45  μm]0.4 [n.a.]NoTissue classification
Leitner et al.26MSI + HSI rigid endoscope [xenon]AOTF400 to 650MSI: 81004×1002 [n.a.]MSI: 0.2NoClassification of cancerous tissue
HSI: 51 [5]
HSI: 1.25 [n.a.]
Baltussen et al.14HSI laparoscope [halogen]2 × push-broom400 to 1000 and 900 to 1700n.a. [3 and 5]1×1312 and 1×320 [n.a.]20 and 30 [60]NoLG CA
Han et al.27HSI flexible endoscope [xenon]Filter wheel405 to 66527 [10]582×752 [n.a.]4.2 [n.a.]NoLG CA
Kumashiro et al.15HSI flexible endoscope [xenon]Push-broom405 to 75070 [5]640×480 [n.a.]30 [60]YesLG CA
Regeling et al.28HSI flexible endoscope [n.a.]Monochromator390 to 68030 [10]1388×1040 [n.a.]6 to 9 [30]NoHN CA
Yoon et al.16Flexible HSI endoscope [LED, halogen]Manual line scanning450 to 710 (due to light source)75 [9] at 18  μm slit; 50  lines/mmn.a. [120  μm at 5 mm distance]0.05/line [0.85/line]YesUG CA
Presented systemHSI laparoscope [LED, xenon, halogen]Push-broom500 to 1000100 [5]640×480 [320  μm at 50-mm distance and 500 nm]4.6 [4.6]YesUG CA
AOTF, acousto-optical tunable filter; LCTF, liquid crystal tunable filter; SRDA, spectrally resolving detector array; U/LG CA, detection of cancers in the upper/lower gastrointestinal tract; and HN CA, head and neck cancer detection.

aSimultaneous acquisition of HSI/MSI and color video (>15  fps).

Here, the presented laparoscopic system provides a high-resolution color video and simultaneous HSI with high spatial and spectral resolution in the visible and near-infrared range. To the best of our knowledge, it is the first system with these capabilities. Furthermore, the typical limitation of pushbroom scanning is overcome for the first time in a visible and near-infrared laparoscope. Moving the spectrograph with a miniaturized stepper motor inside the housing of the laparoscopic camera allows HSI without moving the sample or the laparoscope. This paper aims to describe an HSI laparoscope with high practicability for clinical use and to evaluate it with different light sources. Additionally, HSI was performed on a human tissue resectate with the presented system and a previously described hyperspectral camera for open surgery for comparison.17

2.

Methods

The HSI laparoscope was evaluated in a laboratory set-up with different light sources and variable working distances to the measured reference objects. Furthermore, data from human tissue resectates were acquired with the proposed system and a commercially available HSI system for comparison.

2.1.

Description of the HSI Laparoscope

The overall system is composed of an HSI and color video camera system combined in a three-dimensional (3-D) printed housing with dimensions 8×6×6  cm3 and a rigid laparoscope (HOPKINS® 8711 AGA, KARL STORZ SE & Co. KG, Tuttlingen, Germany) with a 10-mm diameter suitable for white light, near-infrared fluorescence, and autofluorescence imaging. The laparoscope is attached to an objective lens (OL) adapter with a focal length of 14 mm and a C-mount connection on the camera side. The weights of the camera and the used laparoscope (with OL adapter) are 259 and 254 g, respectively. For illustration of the miniaturized camera system components, see Fig. 1. A beam splitter (BS) is partially reflecting the incident visible light only, while the remaining light is transmitted. The reflected light is captured by a color image sensor (IMX290LQR-C, Sony, Tokyo, Japan) with 1920×1080 maximum number of effective pixels at 120 frames per second (fps). After the transmitted light went trough a high-pass filter glass (FG) (GG 495, Schott AG, Mainz, Germany), only light above 500 nm is passing the entrance slit of the spectrograph. Two achromatic lenses are used for focusing and a transmission grating splits up the light that is captured by a monochrome camera sensor (IMX290LLR-C, Sony, Tokyo, Japan). This sensor has a relatively high quantum efficiency in the near-infrared range and can capture 120 fps with full high-definition (HD) resolution. The electronic components of the camera are controlled by a system on module (i.MX8M Quad 1.5 GHz ARM Cortex-A53, NXP Semiconductors, Eindhoven, Netherlands) and the acquired raw images are transferred via gigabit ethernet to a Windows machine (2.11 GHz Intel Core i7) for processing. During scanning, 960×780  pixels (spatial×spectral) of the maximum 2048×1088  pixels of the image sensor are read out at 140 fps. After spectral calibration of the spectrograph with a krypton lamp once, each of the 780 pixels in the spectral dimension can be assigned to one of 500 channels between 500 and 1000 nm. This 960×500 raw image is binned to 480×100 (spatial×spectral) pixels for noise reduction by averaging two pixels in the spatial dimension and five pixels in the spectral dimension. A detailed description of the used calibration and balancing process can be found by Holmer et al.18 The spectrograph is guided by a linear rail to ensure the axis alignment while being moved by an attached stepper motor with a diameter of 15 mm. For the acquisition of a hyperspectral data cube, the spectrograph is moved over the image plane by the stepper motor while 640 images are recorded. This results in a three-dimensional array with 640×480  pixels in the spatial dimension and 100 spectral channels with 5-nm resolution obtained in 4.6 s. Image sizes up to 1920×1080  pixels in the spatial dimension are possible, but require longer acquisition time. Data processing (from raw data to a false-color image) and scanning are performed in parallel, and therefore, no additional time is needed for computation. Furthermore, real-time visualization of the processed false-color image is available during scanning. The field of view (FOV) and the spatial resolution are dependent on the distance to the object and the focal length of the OL. The FOV was measured for 5- and 10-cm distances, and this yielded 55×48  mm2 and 100×83  mm2, respectively.

Fig. 1

Schematic description of the laparoscopic hyperspectral camera system. The light of an external light source is coupled into the laparoscope for object illumination. An OL adapter focuses on different working distances and connects the laparoscope with the camera housing. The incident light of the visible spectral range is partially reflected from a BS on a rgbS. The remaining part of the visible light and all near-infrared light passes the BS and the high-pass FG toward the entrance slit of the spectrograph. The spectrograph consists of two achromatic lenses (L1 and L2), a transmission grating, and a monochromatic camera sensor (bwS). A stepper motor is moving the spectrograph during HSI (push-broom scanner).

JBO_25_8_086004_f001.png

Even though a color image from the rgbS and a false-color image calculated from spectral data show different areas of the object, the arrangement of both systems inside the camera is fixed. This allows the registration of both images after a one-time calculation of the homography. For this purpose, 25 corresponding points are manually annotated in both images. These points are the input for the OpenCV function findHomography19 that uses RANSAC20 to calculate the homography (3×3 transformation matrix) between two planes by minimizing the back-projection error. Once the homography is estimated, the images are registered according to

Eq. (1)

D(x,y)=S(H11x+H12y+H13H31x+H32y+H33,H21x+H22y+H23H31x+H32y+H33),
where D(x,y) are the new pixel coordinates of the false-color image, S(x,y) are the old coordinates, and H is the homography.

2.2.

Light Sources

One commercially available light source: (a) halogen 3900 (Illumination Technologies, Liverpool, USA) as well as two customized ones: (b) Xenon 300 without near-infrared filter and (c) power LED 175 NIR [(b) and (c), KARL STORZ SE & Co. KG, Tuttlingen, Germany] were used for object illumination. The light source (c) emits in the visible (similar to Power LED 175) and near-infrared wavelength ranges. The intensity distribution of the three different light sources and white references were measured under the same conditions. At first, the exposure time and gain of the bwS were adjusted at centered entrance slit position and 5-cm object distance with the LED NIR light source at 80% of its maximum power. Exposure time and gain were constant for all measurements. Second, the power of each light source was increased until the measured intensities at 650 nm were equal for all three light sources. This resulted in an adjusted light source power of 40% for LED NIR, 6% for xenon, and 50% for halogen. During fixed motor position, the bwS provides 480 spectra in the spatial dimension. The mean intensity is calculated from the 100 centered spectra after subtraction of the black level. Moreover, the signal-to-noise ratio (SNR) of the laparoscopic system was estimated at a 5-cm object distance with the LED NIR (power: 80%) and xenon (power: 6%) light source. Similarly, the SNR of the HSI system for open surgery was estimated at a 50-cm object distance with a centered entrance slit position. The SNR was calculated according to

Eq. (2)

SNR=20×logI1000¯|II1000¯|,
where I1000¯ is the average of 1000 acquired raw images and I is one single image. The resulting 480×100 (spatial×spectral) SNR array was averaged in the spatial dimension to yield the SNR over the wavelength for the sensor border (line 0 to 10), center (line 240 to 250), and all lines.

2.3.

Reference Objects and Human Tissue Resectate

For white and color references the ColorChecker (X-Rite, Grand Rapids, Michigan), white paper and a tissue phantom with scattering properties similar to human tissue were used. The latter consists of silicone and barium sulfate in a ratio of 30:7 without additional absorbers. Figure 2 shows the laboratory set up with the laparoscope, light source, and white references. Different realistic object distances were evaluated for the laparoscopic HSI system at 5- to 10-cm distances in 1-cm steps. The spatial resolution was evaluated with the 1951 United States Air Force (USAF) resolution test chart at 5- and 10-cm object distances by calculating the Michelson contrast of the resolution elements at 500 and 800 nm. A sine curve was fitted to the intensity profile and the maximum and minimum intensities for the determination of the Michelson contrast were obtained. The used Michelson contrast cut off was 20%. Furthermore, measurements were performed during abdomino-thoracic esophageal resection in a patient with Barrett’s Esophagus (Siewert classification: adenocarcinoma of esophagogastric junction type I-II, neoadjuvant chemotherapy). Therefore, the resectate was placed in the laparoscopic trainer Laparo Advance (Laparo, Wroclaw, Poland) and the laparoscope was attached to a holding arm during HSI. The ambient light was switched off during all measurements to avoid spectral artifacts.

Fig. 2

Setup for measurements with the (1) laparoscopic hyperspectral camera, (2) rigid laparoscope, (3) custom near-infrared LED light source, (4) white tissue phantom, and (5) ColorChecker. Working distance (d) is controlled manually by moving the mount of (4) and (5) along the guide rail.

JBO_25_8_086004_f002.png

2.4.

Reference HSI System for Open Surgery

The data acquired with the laparoscopic HSI system were compared to the results obtained with a commercially available medical HSI system (TIVITA® Tissue, Diaspective Vision GmbH, Am Salzhaff, Germany). This system was previously described by Kulcke et al.17 and is not suitable for endoscopic use. However, both HSI systems use pushbroom scanning and produce three-dimensional arrays with the same size, spectral resolution, and wavelength range. This allows the calculation of false-color images representing physiologic tissue parameters. Holmer et al.18 introduced the tissue hemoglobin index for skin that is multiplied by 0.5 to derive the organ hemoglobin index (OHI). OHI false-color images were calculated from the spectral data of a human tissue resectate to quantify the hemoglobin content from 0 to 100. The TIVITA camera was placed at a 50-cm distance to the object and six integrated halogen spots were used for illumination. For the evaluation of spectral comparability, reflectance spectra of a ColorChecker target were recorded with the TIVITA and the laparoscopic HSI system at 50-and 10-cm measurement distances, respectively. To get realistic results, the same camera settings as for the SNR measurements were used and were not adjusted for the larger object distance.

2.5.

Data Preprocessing

The reflectance IR of the acquired spectral data was determined for every pixel (x,y) and wavelength (λ) according to

Eq. (3)

IR=(I0Idark)(IwhiteIdark),
where Idark is the dark pattern without illumination, I0 is the acquired intensity, and Iwhite is the intensity measured on a white reference. Iwhite was determined for every used object distance on white paper. The absorbance A was calculated by A=log10(IR).

For the calculation of OHI, the following equation was used:

Eq. (4)

OHI=0.5×A¯530590  nmA¯785825  nmaba,
where A¯ is the average of the absorbance in the given wavelength range. The calibration values a and b are used to scale the OHI in the range from 0 to 100.

3.

Results

Standardized intensities of the three different light sources and white references are shown in Fig. 3(a). Ripples in the measured intensities above 800 nm are due to the spectral efficiency of the sensor and corrected during white balancing. The spatial intensity decrease of the light sources was evaluated during the same measurement from the illumination spot center to the border of the image. An intensity drop about 50% at 150 pixels distance from the spot center was observed for the LED NIR (575 nm), xenon and halogen (685 nm) light source [Fig. 3(b)]. The mean SNR of all sensor lines was 30 to 43 dB in the spectral range from 500 to 950 nm as illustrated in Figs. 3(c) and 3(d). Due to the intensity decrease at the image borders, up to 7 dB difference in the spatial dimension was observed. The TIVITA system with halogen lighting unit (50-cm object distance) showed an overall higher SNR compared to the laparoscopic system with LED or xenon light source. However, in the spectral range with high absorption coefficients of hemoglobin (<600  nm), the LED NIR light source provided a higher SNR.

Fig. 3

Spectral characteristic of the laparoscopic HSI system for different light sources (5-cm object distance): (a) standardized intensity of the LED NIR, xenon, and halogen light sources for three different white references (dotted lines represent the ColorChecker and dashed lines represent the white phantom). (b) Intensity decrease from illumination spot center to the border of the image for LED NIR (575 nm), xenon and halogen (685 nm). (c) SNR of the laparoscopic HSI system with LED NIR light source compared to the TIVITA system. The mean SNR of sensor lines 0 to 10, 240 to 250, and all lines in the spatial dimension are plotted for both systems. (d) Mean SNR of all sensor lines for the laparoscopic HSI (LED NIR and xenon light source) compared to the TIVITA system (halogen light).

JBO_25_8_086004_f003.png

The spectral distribution of the LED NIR light source measured at 5- to 10-cm distances in 1-cm steps is shown in Fig. 4. An intensity decrease proportional to 1/distance1.8 was observed, which approximately follows the inverse-square law for point sources.

Fig. 4

Intensity decrease in the image center measured with the laparoscopic HSI system and LED NIR light source at different object distances: (a) the measured intensities are approximately following the inverse-square law (intensity1/distance2). (b) Logarithmic scale and normalized intensities indicate that the shape of the spectra is distance-independent.

JBO_25_8_086004_f004.png

At a 5-cm object distance and 500 nm, the spatial resolution was 320  μm in the x dimension and 160  μm in the y dimension. The spatial resolution approximately halved (630  μm in the x dimension and 315  μm in the y dimension) for a doubled object distance (10 cm). A decreased spatial resolution was observed in the near-infrared range (800 nm) for the y dimension (500  μm) though it remained constant for the x dimension (scanning direction).

In Fig. 5, a static image of the color video from the color image sensor (rgbS), a color image calculated from HSI data of the bwS, and the registration of both images with semitransparent overlay are shown. The average root-mean-square error (RMSE) of the reflectance (500 to 900 nm) from 11 ColorChecker patches measured with the laparoscopic and TIVITA system was 0.03 (±0.02). Three exemplary reflectance curves of these patches are given in Fig. 5(d) for both systems.

Fig. 5

Simultaneous video and HSI of the ColorChecker target: (a) video from the color image sensor (rgbS). (b) Registration and semitransparent overlay of the rgbS video and color image from HSI after one-time calculation of the homography. (c) Color image calculated during hyperspectral data acquisition. Pseudocolors are calculated from the measured reflectance between 530 and 725 nm, due to the lower SNR of the TIVITA system below this range. (d) Reflectance of three exemplary ColorChecker patches measured with the laparoscopic HSI and TIVITA system. Error bars indicating the standard deviations for the patches recorded with the laparoscopic system. Standard deviations for the TIVITA system are <2% and omitted for clarity.

JBO_25_8_086004_f005.png

The laparoscopic HSI system was successfully used in clinical set up for measurements of a resected adenocarcinoma of the esophagogastric junction from a patient with Barrett’s Esophagus. Data were acquired from extra- and intraluminal immediately after abdomino-thoracic esophageal resection. The corresponding structures in the color images acquired with the TIVITA and laparoscopic HSI system were matched postoperatively. Figure 6 shows the color images and OHI false-color images from the TIVITA and the laparoscopic HSI system. In addition, the mean absorbance spectra inside a corresponding ROI are illustrated. The data are plotted from 500 to 960 nm for the laparoscopic system and 550 to 1000 nm for TIVITA because of the low intensities outside this range. The RMSE of the two mean absorbance spectra inside the region of interest (ROI) is 0.02 (550 to 960 nm). For eight randomly selected ROIs, the average RMSE is 0.09 (±0.05).

Fig. 6

Intraluminal HSI of the resected adenocarcinoma of the esophagogastric junction: color image and OHI calculated from spectral data acquired with (a), (b) TIVITA and (c), (d) laparoscopic HSI system. The corresponding FOV is labeled as a blue circle. (e) Mean absorbance spectrum of the pixels inside the black square for both HSI systems. (f) Mean absorbance spectrum of adenocarcinoma (pixels inside yellow circle), esophageal and gastric mucosa (outside of the FOV) measured with the laparoscopic HSI system and labeled by a surgeon according to visual and haptic perception.

JBO_25_8_086004_f006.png

4.

Discussion

The intraoperative estimation of tissue perfusion and differentiation is essential during most laparoscopic procedures. HSI can support visual decision making, but is not yet ready for routine laparoscopic use. An HSI system with simultaneous high-resolution video for minimal-invasive surgery may reduce the risk of perfusion related complications and assist the identification of lesions and risk structures.

In this work, a laparoscopic system capable of spectral imaging from 500 to 1000 nm with high spatial and spectral resolution in 4.6 s was described. The raw spectral data are processed and visualized in real time during acquisition, hence no additional time is needed. Furthermore, the handheld device provides high-resolution video during HSI and allows pushbroom imaging without the previously reported limitations of moving the object or the laparoscope. The registration of false-color images derived from the hyperspectral data (e.g., perfusion maps) and high-resolution video provides augmented visual information. The measured ColorChecker reflectance data were consistent with a commercially available medical HSI device for the practical object distance of 10 cm. Moreover, corresponding regions of the resected esophagogastric junction showed the same spectral signature for both systems although the light source and measurement distance were different.

Additionally, OHI false-color images of the resected tissue from the open and laparoscopic HSI system showed the same qualitative results. Quantitative deviations of OHI were observed and could be due to the time delay between measurements, repositioning in the laparoscopic trainer, and glare artifacts, which needs to be investigated more closely.

Only minor differences were observed for the spectral shape of the three different white references, however, variations in absolute intensity were present and must be considered when comparing reflectance values. Even though the used light sources slightly differed in their spatial distribution, all showed a strong intensity decline in the FOV. A homogenous illumination would improve the SNR at the image borders.

The entrance slit can be moved automatically by the integrated motor to scan the image or can be stopped at any position if desired. This design also allows manual scanning, such as that recently described by Yoon et al.16 for flexible endoscopy.

Currently, the SNR in the near-infrared range is limited by the available light sources and the transmission of the used laparoscopes. Work is in progress to solve this problem. The size of the system is already comparable to usual laparoscopic cameras. However, the housing could become even smaller by removing parts of the processing hardware. To avoid motion artifacts, the laparoscope was attached to a holding arm during scanning. Movements of the tissue, for instance due to breathing, cannot be prevented and thus should be aimed at further reduction of HSI acquisition time.

In the authors’ opinion, rigid laparoscopic HSI has great potential for fast and repeatable quantification of tissue perfusion as reported by Clancy et al.21 Although the measured reflectance data agreed with the results from the TIVITA system, studies on perfused tissue need to verify that the approved perfusion indices are valid for laparoscopic use.

Laparoscopic HSI in the visible and near-infrared range with short processing time and simultaneous video is feasible and fulfills the requirements for routine clinical use. Studies with more clinical data are needed to evaluate the benefits of laparoscopic HSI for intraoperative applications.

5.

Conclusions

An HSI system with high spatial and spectral resolution in the visible and near-infrared range and simultaneous color video for minimally invasive surgery was presented. Reference objects and resected tissue were used to show the consistency of the measured data from the laparoscopic HSI and the approved TIVITA system. The compact handheld design, rapid HSI, and high-resolution video at the same time enable high clinical practicability. This work may support the translation of HSI to routine laparoscopic surgery. Future studies on the added value of laparoscopic HSI for the patient outcome are needed.

Disclosures

The laparoscopic hyperspectral camera used for the measurements in this publication was developed by Diaspective Vision GmbH. H. Köhler is an employee and A. Kulcke is the CEO of this company. In the long term, Diaspective Vision has a proprietary interest in the development of the laparoscopic hyperspectral camera resulting in a product for routine clinical use. All other authors have no financial interests and financial arrangements with Diaspective Vision GmbH and have received no funding for the measurements and/or preparation of this manuscript. The light sources used during the measurements were provided by KARL STORZ SE & Co. KG. Ethical approval: Experimental hyperspectral measurements from patients for the evaluation of the new technology have obtained the ethics approval by the Ethics Committee of the University Leipzig under 026/18-ek. The study was conducted according to the Declaration of Helsinki.

Acknowledgments

The authors would like to acknowledge financial support by the Federal Ministry of Education and Research of Germany (Project No. 13GW0248B), the German Research Foundation (DFG), and Leipzig University within the Program of Open Access Publishing.

References

1. 

Q. Li et al., “Review of spectral imaging technology in biomedical engineering: achievements and challenges,” J. Biomed. Opt., 18 (10), 100901 (2013). https://doi.org/10.1117/1.JBO.18.10.100901 JBOPFO 1083-3668 Google Scholar

2. 

B. Jansen-Winkeln et al., “Determination of the transection margin during colorectal resection with hyperspectral imaging (HSI),” Int. J. Colorectal Dis., 34 (4), 731 –739 (2019). https://doi.org/10.1007/s00384-019-03250-0 IJCDE6 1432-1262 Google Scholar

3. 

M. Barberio et al., “HYPerspectral enhanced reality (HYPER): a physiology-based surgical guidance tool,” Surg. Endosc., 34 1736 –1744 (2020). https://doi.org/10.1007/s00464-019-06959-9 Google Scholar

4. 

D. Nouri, Y. Lucas and S. Treuillet, “Hyperspectral interventional imaging for enhanced tissue visualization and discrimination combining band selection methods,” Int. J. Comput. Assist. Radiol. Surg., 11 (12), 2185 –2197 (2016). https://doi.org/10.1007/s11548-016-1449-5 Google Scholar

5. 

M. Maktabi et al., “Classification of hyperspectral endocrine tissue images using support vector machines,” Int. J. Med. Rob., e2121 (2020). https://doi.org/10.1002/rcs.2121 Google Scholar

6. 

D. Yudovsky et al., “Monitoring temporal development and healing of diabetic foot ulceration using hyperspectral imaging,” J. Biophotonics, 4 (7–8), 565 –576 (2011). https://doi.org/10.1002/jbio.201000117 Google Scholar

7. 

A. Holmer et al., “Hyperspectral imaging of the degradation of meat and comparison with necrotic tissue in human wounds,” J. Spectr. Imaging, 8 a9 (2019). https://doi.org/10.1255/jsi.2019.a9 Google Scholar

8. 

M. Halicek et al., “In-vivo and ex-vivo tissue analysis through hyperspectral imaging techniques: revealing the invisible features of cancer,” Cancers, 11 (6), 756 (2019). https://doi.org/10.3390/cancers11060756 Google Scholar

9. 

S. G. B. de Koning et al., “Toward assessment of resection margins using hyperspectral diffuse reflection imaging (400–1,700 nm) during tongue cancer surgery,” Lasers Surg. Med., 52 (6), 496 –502 (2019). https://doi.org/10.1002/lsm.23161 LSMEDI 0196-8092 Google Scholar

10. 

M. Halicek et al., “Tumor detection of the thyroid and salivary glands using hyperspectral imaging and deep learning,” Biomed. Opt. Express, 11 (3), 1383 (2020). https://doi.org/10.1364/BOE.381257 BOEICL 2156-7085 Google Scholar

11. 

S. Stewart et al., “Visible near infrared reflectance molecular chemical imaging of human ex vivo carcinomas and murine in vivo carcinomas,” J. Biomed. Opt., 25 (2), 026003 (2020). https://doi.org/10.1117/1.JBO.25.2.026003 JBOPFO 1083-3668 Google Scholar

12. 

J. Shapey et al., “Intraoperative multispectral and hyperspectral label-free imaging: a systematic review of in vivo clinical studies,” J. Biophotonics, 12 (9), e201800455 (2019). https://doi.org/10.1002/jbio.201800455 Google Scholar

13. 

A. S. Luthman et al., “Bimodal reflectance and fluorescence multispectral endoscopy based on spectrally resolving detector arrays,” J. Biomed. Opt., 24 (3), 031009 (2018). https://doi.org/10.1117/1.JBO.24.3.031009 JBOPFO 1083-3668 Google Scholar

14. 

E. J. M. Baltussen et al., “Hyperspectral imaging for tissue classification, a way toward smart laparoscopic colorectal surgery,” J. Biomed. Opt., 24 (1), 016002 (2019). https://doi.org/10.1117/1.JBO.24.1.016002 JBOPFO 1083-3668 Google Scholar

15. 

R. Kumashiro et al., “Integrated endoscopic system based on optical imaging and hyperspectral data analysis for colorectal cancer detection,” Anticancer Res., 36 (8), 3925 –3932 (2016). ANTRD4 0250-7005 Google Scholar

16. 

J. Yoon et al., “A clinically translatable hyperspectral endoscopy (HySE) system for imaging the gastrointestinal tract,” Nat. Commun., 10 (1), 1902 (2019). https://doi.org/10.1038/s41467-019-09484-4 NCAOBW 2041-1723 Google Scholar

17. 

A. Kulcke et al., “A compact hyperspectral camera for measurement of perfusion parameters in medicine,” Biomed. Tech., 63 (5), 519 –527 (2018). https://doi.org/10.1515/bmt-2017-0145 Google Scholar

18. 

A. Holmer et al., “Hyperspectral imaging in perfusion and wound diagnostics—methods and algorithms for the determination of tissue parameters,” Biomed. Eng. Biomed. Tech., 63 (5), 547 –556 (2018). https://doi.org/10.1515/bmt-2017-0155 Google Scholar

19. 

B. Gary and K. Adrian, “Learning OpenCV: computer vision with the OpenCV library,” Newton, Massachusetts (2018). Google Scholar

20. 

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM, 24 (6), 381 –395 (1981). https://doi.org/10.1145/358669.358692 CACMA2 0001-0782 Google Scholar

21. 

N. T. Clancy et al., “Intraoperative measurement of bowel oxygen saturation using a multispectral imaging laparoscope,” Biomed. Opt. Express, 6 (10), 4179 (2015). https://doi.org/10.1364/BOE.6.004179 BOEICL 2156-7085 Google Scholar

22. 

Y. Fawzy, S. Lam and H. Zeng, “Rapid multispectral endoscopic imaging system for near real-time mapping of the mucosa blood supply in the lung,” Biomed. Opt. Express, 6 (8), 2980 (2015). https://doi.org/10.1364/BOE.6.002980 BOEICL 2156-7085 Google Scholar

23. 

M. Hohmann et al., “In-vivo multispectral video endoscopy towards in-vivo hyperspectral video endoscopy,” J. Biophotonics, 10 (4), 553 –564 (2017). https://doi.org/10.1002/jbio.201600021 Google Scholar

24. 

S. S. More, J. M. Beach and R. Vince, “Early detection of amyloidopathy in Alzheimer’s mice by hyperspectral endoscopy,” Invest. Opthalmol. Visual Sci., 57 (7), 3231 (2016). https://doi.org/10.1167/iovs.15-17406 Google Scholar

25. 

Y. Zhang et al., “Tissue classification for laparoscopic image understanding based on multispectral texture analysis,” J. Med. Imaging, 4 (1), 015001 (2017). https://doi.org/10.1117/1.JMI.4.1.015001 JMEIET 0920-5497 Google Scholar

26. 

R. Leitner et al., “Multi-spectral video endoscopy system for the detection of cancerous tissue,” Pattern Recognit. Lett., 34 (1), 85 –93 (2013). https://doi.org/10.1016/j.patrec.2012.07.020 PRLEDG 0167-8655 Google Scholar

27. 

Z. Han et al., “In vivo use of hyperspectral imaging to develop a noncontact endoscopic diagnosis support system for malignant colorectal tumors,” J. Biomed. Opt., 21 (1), 016001 (2016). https://doi.org/10.1117/1.JBO.21.1.016001 JBOPFO 1083-3668 Google Scholar

28. 

B. Regeling et al., “Hyperspectral imaging using flexible endoscopy for laryngeal cancer detection,” Sensors, 16 (8), 1288 (2016). https://doi.org/10.3390/s16081288 SNSRES 0746-9462 Google Scholar

Biography

Hannes Köhler received his BS and MS degrees in medical engineering from the Technical University of Ilmenau in 2014 and the University of Lübeck in 2017, respectively. He is a research fellow at the Innovation Center Computer Assisted Surgery of the University of Leipzig. His current research focuses on biomedical optics, spectral imaging, and image processing.

Biographies of the other authors are not available.

© The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Hannes Köhler, Axel Kulcke, Marianne Maktabi, Yusef Moulla, Boris Jansen-Winkeln, Manuel Barberio, Michele Diana, Ines Gockel, Thomas Neumuth, and Claire Chalopin "Laparoscopic system for simultaneous high-resolution video and rapid hyperspectral imaging in the visible and near-infrared spectral range," Journal of Biomedical Optics 25(8), 086004 (28 August 2020). https://doi.org/10.1117/1.JBO.25.8.086004
Received: 27 April 2020; Accepted: 12 August 2020; Published: 28 August 2020
JOURNAL ARTICLE
13 PAGES


SHARE
Advertisement
Advertisement
Back to Top