Automatic correction of motion artifact in dynamic contrast-enhanced fluorescence imaging during open orthopedic surgery

Abstract. Significance Indocyanine green-based dynamic contrast-enhanced fluorescence imaging (DCE-FI) can objectively assess bone perfusion intraoperatively. However, it is susceptible to motion artifact due to patients’ involuntary respiration and mechanical disturbance. Reducing motion artifacts would significantly improve DCE-FI for orthopedic surgical guidance. Aim Our primary objective is to develop an automated correction method to reduce motion artifacts in DCE-FI and improve the accuracy of bone perfusion assessment. Approach We developed an automated motion correction approach based on frame-by-frame mutual information (MI) and validated the effectiveness of this approach in various phantom studies and patient images from 45 imaging sessions of fifteen amputees. Results The MI-based correction reduced motion artifacts by 93% for mechanical disturbances and 76% for simulated respiration in phantom studies. Patient images show improved alignment, improved kinetic curves, and restored bone perfusion-related parameters with an average correction of 4.3 and 9.6 mm in x- and y-axes per session. Conclusions The automated MI-based motion correction was able to eliminate motion artifacts effectively and significantly improved the quantitative assessment of bone perfusion by DCE-FI.


Introduction
Infection following trauma is one of the most prevalent and challenging complications faced by orthopedic surgeons.Inadequate tissue perfusion plays a critical role in this complication since poorly perfused bone can be a nidus for bacterial biofilm formation creating resistance to antibiotics. 1,2These biofilms make treatment of infections difficult as they increase the bacteria's ability to evade the immune system, share antibiotic resistant genes, and proliferate additional planktonic bacteria. 3Because of this, management of open fractures and fracture-related infection relies on aggressive and thorough debridement in an effort to remove all poorly perfused bone. 4,5However, there is a limited number of imaging tools to objectively inform bone perfusion and guide debridement.
In contrast to other clinical imaging modalities, such as dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) 6,7 and combined positron emission tomography/ computed tomography (PET/CT), 8 indocyanine green (ICG)-based dynamic contrast-enhanced fluorescence imaging (DCE-FI) presents an intraoperative solution.][11] Although DCE-FI shows both feasibility and ease of utilization in the surgical setting, motion artifacts resulting from patient's involuntary respiration and other mechanical disturbances during imaging data acquisition can potentially lead to errors in quantitative bone perfusion assessment.Previous studies have utilized marker tracking and edge detection for automated motion compensation in fluorescence imaging. 12,13][20] While MI offers a better solution for reducing the motion artifact in intraoperative fluorescence imaging, the conventional MI calculation that using a fixed reference may fall short for DCE-FI in intraoperative orthopedic imaging.This is due to the extensive heterogeneity of fluorescence intensities arising from bone's deep three-dimensional structure and coupled with the dynamic changes in perfusion levels that occur across the imaging field over time.To overcome this limitation, a frame-by-frame MI-based motion correction approach for DCE-FI has been developed and validated using tissue phantoms and 45 intraoperative imaging sessions on 15 amputation cases.The results of the motion correction demonstrated a significant reduction in motion artifacts.The alignment of image frames at different time points within each imaging session resulted in improved kinetic curves and enhanced perfusion-related parameters for identifying bone perfusion levels.and utilized 805 nm NIR light for ICG excitation.A charge-coupled device camera with an 820 to 900 nm bandpass filter was employed for fluorescence light imaging.Additionally, to capture white-light images, an RGB camera was connected to the system via a beam splitter [Fig.1(b)].This setup allowed simultaneous capture of both fluorescence and white-light images.

Materials and Methods
During each imaging session, a white-light image was captured first, followed by a fluorescence video recorded at a rate of 3.75 frames per second for 1024 frames (maximum frame number).Each video frame has an 18 × 13.5 cm field of view (FOV) with 1024 × 768 pixels and a bit depth of 8. Further system specifications were described in previous publications. 10,11,21,22

MI-Based Motion Correction
The motion correction is based on maximizing MI, calculated frame by frame.MI is a well-established metric in the field of medical image processing used to assess the similarity of two images. 17It is defined as follows: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 1 ; 1 1 7 ; 5 8 0 MI ¼ −HðA; BÞ þ HðAÞ þ HðBÞ; (1) where HðA; BÞ is the joint entropy of reference image A and target image B, calculated as where p X ðxÞ is the marginal probability density function.In this study, given that the acquired fluorescence images have discrete intensities, p AB ða; bÞ is computed using the joint histogram of grayscale intensities from two unsigned eight-bit images, whereas p X ðxÞ is derived from the sum of the normalized joint histogram on each row or column.
In theory, the maximal MI corresponds to the most geometrically aligned images. 18The maximization of MI between the reference and adjusted target frame is the process of locating their most complex overlapping areas [by maximizing individual entropies p X ðxÞ] that has a strong statistical relationship [by minimizing the joint entropy p AB ða; bÞ]. 23Since the entire image pixels and frames are involved into the MI calculation, manual selection of landmarks or references for image co-registration can be completely eliminated.
The reference frame was constantly updated in our frame-by-frame correction approach.Two adjacent frames were set as the reference (diagonal-line-filled) and the target (dashed line), respectively.The target frame was shifted along the xand y-axes, and MI was calculated for each pair of the reference and translated target frames.The maximal MI value among all possible translation combinations determined the optimal translation, which was applied to the target frame.The corrected target frame was then set as the new reference, and its adjacent frame as the new target in the next run.The process was then repeated until the last frame was reached.

Phantom Validation
To verify the efficacy of our MI-based motion correction in mitigating the mechanical disturbances from the imaging device, we conducted validation experiments using an ICG-equivalent phantom (QUEL Imaging, White River Junction, VT). 24As shown in Fig. 2(a), this phantom containing nine 8-mm wells filled with varying ICG equivalent concentrations, ranging from 1 μM (top left) to 0 (bottom right).To simulate typical mechanical disturbances, we induced random vibrations in the imaging head of the SPY Elite Fluorescence Imaging System by manually tapping the articulating arm [Fig.1(a)].The imaging FOV underwent oscillations for several cycles before returning to a stationary status.Fluorescence imaging of the phantom was conducted over a 90-s duration, with manual tapping at 0, 30, and 60 s, ensuring sufficient time for a vibration to complete.The tapping intensity was varied to assess the power of the correction against different levels of mechanical vibrations.
To further examine the correction's capability in addressing the motions due to the patient's involuntary respiration, we conducted another validation study using the ICG-equivalent phantom placed on the lower leg of a normal subject in a supine position on a table.The subject was instructed to contract and relax leg muscles to mimic breathing motions observed in intraoperative images obtained from patient cases.
The intensity variations due to the motion artifacts in the images were assessed.The areas inside the 100 and 300 nM wells were designated as the regions of interest (ROIs), from which the average ICG intensity of the circular ROI was extracted before and after the correction.

Patient Imaging
This study was conducted in compliance with the Health Insurance Portability and Accountability Act and approved by the institutional review board at Dartmouth Health, Lebanon, NH.We obtained written informed consent from 15 patients who underwent lower extremity amputation.During each imaging session for each patient, we employed ICG-based DCE-FI using a fluorescence imaging system, acquiring images every 0.267 s over a 4.5-min duration.Following 20 s of pre-injection imaging, we intravenously administered 0.1 mg∕kg of ICG to the patient.Throughout the surgery, we recorded three individual imaging sessions before and after two surgical procedures, namely osteotomy and soft tissue-stripping, resulting in 45 evaluable datasets.

Imaging Analysis and Display
Figure 2 presents photos, images, and a schematic diagram that detail the image analysis and display process of a phantom containing nine 8-mm wells filled with varying ICG equivalent concentrations, ranging from 1 μM (top left) to 0 (bottom right), as indicated in Fig. 2(a).As depicted in Fig. 2(b), the fluorescence emitted from the top three and middle two wells with equivalent ICG concentrations of 1 μM, 300 nM, 100 nM, 30 nM, and 10 nM has been successfully detected.In this case, the fluorescence intensity is measured in relative fluorescence units (RFUs).After the MI-based motion correction, we compared images of these two wells captured at various time points to assess frame alignment.Specifically, as shown in Fig. 2(c), green and magenta highlight intensity discrepancies between two overlaid fluorescence images of 300 and 100 nM wells with a 1 mm horizontal displacement, and the gray area indicates a region with a close agreement between two overlaid frames.In Fig. 2(d), we present the dynamic bone blood perfusion-related parameters, 11 including maximum fluorescence intensity (I max ), time-to-peak (TTP), and ingress slope (IS).These parameters were calculated from the average fluorescence intensity curve within each ROI to validate the effectiveness of MI-based approach.Figure 5 demonstrates the joint histograms of the two frames in Fig. 4 before and after MI-based corrections with fixed-reference and frame-by-frame methods.Following the  Figure 6 presents the kinetic curves before and after frame-by-frame motion correction on DCE-FI of the same patient case discussed above.The white-light image exhibits an osteotomy at the middle of the tibia [Fig.6(a)].A fluorescence image captured at one minute after ICG injection is superimposed on a white-light image [Fig.6(b)].Four circular ROIs with a 10 mm diameter-labeled p1 (magenta) and p2 (green) for the proximal region toward the head, and d1 (yellow) and d2 (blue) for the distal region toward the foot-were positioned at equidistant points along the axis from the feet to the head, spanning the entire imaging FOV of the tibias.The kinetic curves corresponding to these ROIs are presented in Fig. 6(c), color-coded to match the ROIs.The dashed and solid lines represent kinetic curves before and after the motion correction, respectively.The motion-induced intensity surges were smoothed out, and the general trend of ICG bolus wash-in and wash-out was restored after correction.

Results
Figure 7 presents the kinetic parameters extracted from the four ROIs in Fig. 6(b) before and after the frame-by-frame correction.Due to the osteotomy damaging endosteal blood supply in distal region, perfusion-related kinetic parameters (I max , IS, and TTP) in ROIs of d1 and d2 were expected to be significantly reduced (I max and IS) or increased (TTP), compared to those in ROIs  of p1 and p2.However, as shown in Fig. 7, before correction, these differences were not evident (left portion of the bar graph in each subfigure).In contrast, after the correction, I max and IS in p1 and p2 (solid line) were higher than those in d1 and d2 (dashed line), whereas proximal TTPs became much lower than those in distal ROIs.These drastic changes in perfusion-related parameters demonstrate a lower perfusion in the distal regions after the motion correction, indicating the successful restoration of kinetic parameters.
The automated MI-based motion correction was applied to all images in 45 imaging sessions (baseline, osteotomy, and circumferential periosteal/soft tissue stripping) of 15 amputation cases.Figure 8 shows the total corrected displacements calculated by summing the absolute values of correction between each frame in xand y-axes of each imaging session.The range of total displacements corrected varied from 0 to 23.9 mm [mean: 4.3 mm; standard deviation (SD): 5.3 mm] and 0 to 50.8 mm (mean: 9.6 mm; SD: 13.8 mm) along the xand y-axes, respectively.The average displacement in y-axis was more than double of that in x-axis, indicating more substantial motion in y-direction in this amputation cohort.

Discussion
In addition to clinical imaging modalities such as DCE-MRI 6,7 and PET/CT, 8 other imaging modalities such as photoacoustic imaging, 25,26 laser speckle contrast imaging, 27 and laser Doppler flowmetry 28 have been adapted for in vivo blood flow assessment across various medical applications.Photoacoustic imaging stands out by providing superior surface spatial resolution and capturing angiographic images of tissue.However, it relies on a high-power, short-pulse, and wavelength-tunable laser source to irradiate tissue, and it uses multiple ultrasound transducers to detect the resulting pressure waves.Nevertheless, this approach has limitations related to imaging depth, laser power safety considerations, and the complexities and costs associated with solid-state laser systems, which hinder its widespread clinical adoption for assessing bone  perfusion in real patients. 29Additionally, its imaging accuracy can be uncertain because the ultrasound signal is affected by the complexities of light absorption and scattering under multiwavelength illumination. 29Laser speckle contrast imaging, on the other hand, can effectively capture relative blood flow in relatively clean and flat tissue surfaces based on speckle pattern analysis.However, it faces challenges when applied to intraoperative orthopedic imaging due to the surface roughness and deep 3D curvature of bones during surgery.Laser Doppler flowmetry is an invasive technique that provides blood flow information at specific location, but it does not be suitable in conditions where contact with the study area is avoided such as surgical interventions. 30In contrast, as shown in this study, although ICG-based DCE-FI relies on external contrast agents to generate useful images, it can capture the varies blood perfusion level of the human bone due to the different levels of the damage intraoperatively, to realize the surgical guidance for orthopedic surgery.
Various techniques have been developed to remove motion artifact in dynamic imaging, 31,32 including normalized MI, which reduces the impact of overlapping areas, [33][34][35] and regional MI, which integrates spatial information of pixel intensity distribution. 23,36,37Comparing these approaches, our simple frame-by-frame MI-based approach developed in this study is particularly advantageous for intraoperative DCE-FI in orthopedic surgery due to several reasons.First, it demonstrates robustness to clinical interferences and eliminates the need for any additional hardware or manual selection.Second, it effectively addresses the high heterogeneity of fluorescence intensity across the entire bone area within the imaging FOV, resulting in improved image processing stability.
Due to the large number of imaging frames that require processing for each case, and the limited processing time allowed intraoperatively, we have made some assumptions and simplifications to reduce processing time while maintaining effectiveness.Given the rigid nature of bone, we assumed that the corrections can be achieved through rigid transformation.Additionally, we assume that all motions occurred at the same depth and were limited to planar translations, neglecting minor motions such as rotation and movements in depth for this application.Moreover, temporal down-sampling at an appropriate level was employed to accelerate the processing time and reduce accumulated errors in the frame-by-frame registration process.
For the analysis presented in Figs. 6 and 7, ROIs were designated to optimally represent perfusion across the entire tibial region of each patient under varying bone conditions while ensuring methodological consistency and avoiding a biased selection that would only showcase the most significant or dramatic improvements.The imaging FOV of the tibia was evenly divided into four sections from the feet toward the head, with a 10 mm diameter ROI centrally located in each section.Two ROIs were placed on the distal end and two on the proximal end, respectively.This arrangement accounts for the average coronal dimensions of the tibia and helps to mitigate potential edge artifacts.These ROIs are indicative of the system's general performance in capturing key perfusion metrics, such as I max , IS, and TTP.Additionally, we extracted and compared the average time-intensity curves of the tibia's entire proximal and distal halves, both pre-and postcorrection.The trends observed corroborate the improvements postcorrection, similar to those depicted in Fig. 6(c).The utilization of four smaller ROIs, as opposed to two larger ones, yielded more precise insights into the correction's efficacy across varying perfusion levels within the tibia.
As observed in Fig. 8, the corrected displacements were significantly reduced in the later cases (4 to 15) compared to the early cases (1 to 3).This improvement can be attributed to the enhanced surgical techniques adopted in subsequent cases.Specifically, surgeons took greater care to avoid nonessential clinical interventions during imaging sessions, such as minimizing movement of retractors and carefully suctioning away blood from the surface with minimal tissue contact.
In Fig. 3, we assess the effectiveness of MI-based motion correction for minimizing motion artifacts caused by mechanical disturbances and patient respiration.The analysis focused on wells with 100 and 300 nM ICG-equivalent concentrations, selected based on their fluorescence intensities matching those observed in the tibia via the SPY Elite Fluorescence Imaging System and correlating with the extent of bone damage.Figure 9 presents a comparison of ICG intensity in the tibial bone area of 15 patient's postlower extremity amputation against that in phantom wells with equivalent ICG concentrations.Figure 9(a) shows the average histogram distributions across various imaging sessions: baseline (top), after osteotomy (middle), and following circumferential periosteal/soft tissue stripping (bottom), with 90% confidence intervals shaded.The diagonal lines mark the area within the modified full width at half maximum (mFWHM) 38 of the average histograms.As depicted in Fig. 9(a), the ICG intensities within the mFWHM range from 18 to 176 RFUs.Using the calibration curve of fluorescence intensity versus ICG-equivalent concentrations from the validation phantom in Fig. 9(b), these ICG intensity ranges correspond to concentrations between 49 and 540 nM.
To illustrate our approach, we focused on case 2, where a significant amount of motion artifacts was observed.The comparison of boundary alignment of bone to soft tissue (Fig. 4) provides a straightforward and compelling demonstration of the improvement in imaging quality.
Furthermore, the changes in joint histograms before and after the correction (Fig. 5) demonstrate the efficacy of the MI-based approach, especially with the frame-by-frame reference approach.
Quantitative analysis results in Figs. 6 and 7 revealed that the kinetic behaviors of tibial tissues were restored after the removal of motion artifacts.Although there are no standard perfusion references corresponding to the damages caused by osteotomy or soft tissue stripping, we assumed that the distal area would be less perfused than the proximal area after osteotomy.This assumption is based on the original higher vascularization in the proximal half of the tibia, 39 and the subsequent reduction in distal perfusion due to the osteotomy, which cut off both periosteal and endosteal blood supply toward the distal region.The precorrection levels of perfusion indicated the significance of motion artifact removal in DCE-FI to achieve intraoperative and objective perfusion assessments.
For our phantom validations, we simulated the two most common types of motion encountered in intraoperative DCE-FI imaging during orthopedic surgery, namely, mechanical disturbances and breathing motions.While our imaging system operates at a video frame rate of 3.75 Hz, which effectively captures breathing artifacts at approximately 0.3 Hz, it falls short in adequately capturing high-frequency disruptive mechanical motions that can occur within a single frame.Additionally, even though our correction process is applied on a frame-by-frame basis and incorporates permanent displacements as the cumulative sum of interframe displacements, it is essential to acknowledge that as these displacements accumulate during the correction process, significant permanent displacement can lead to a portion of the bone area shifting out of the FOV, potentially causing the loss of information in that area due to the correction process.Nonetheless, the results presented in Fig. 3 demonstrate that our method effectively mitigates high-frequency artifacts to an acceptable level.
Our optimization involves searching for the maximum MI using a kernel of 3 × 3 pixels instead of an exhaustive grid search.With this optimized correction between two frames, the registration can be achieved within seconds.However, since the approach needs to be implemented frame-by-frame and each dataset contains 1024 frames, the total processing time for an entire dataset is ∼20 min.Reflecting on our previous success with utilizing parallel processing to markedly reduce DCE-FI processing times from 20 min in Matlab to under a minute using Python's multiprocessing package, applying a similar strategy to MI calculations will enable the real-time image registration feedback.This advancement could potentially cut down the time required to provide image guidance in orthopedic surgeries to a few minutes. 40,41

Conclusion
The results from this study demonstrate that automatic MI-based correction reduces motion artifact during the DCE-FI imaging data acquisition and significantly improve the quantitative assessments of bone perfusion.

Disclosures
The authors declare that there are no conflicts of interest related to this article.

2. 1
Fig. 1 (a) Fluorescence imaging system and (b) imaging setup for open orthopedic surgery.

Figure 3
Figure 3 demonstrates the efficacy of motion correction based on MI for mitigating motion artifacts caused by mechanical disturbances and patient respiration.Given the partial fluorescence

Fig. 2
Fig. 2 Photos, images, and a schematic diagram of the image analysis and display using a phantom as an example.(a) Photo and (b) fluorescence image of the phantom.Nine 8-mm wells are filled with silicon material with various ICG-equivalent concentrations ranging from 1 μM (top left) to 0 (bottom right).(c) Overlaid image of two fluorescence frames with a 1 mm horizontal displacement for 300 nM (top) and 100 nM wells (bottom), with intensity differences highlighted by green and magenta.(d) Schematic diagram of the kinetic parameters for assessing bone perfusion.

Figure 4
displays the overlay of two tibial fluorescence images captured at 150 and 250 s, respectively.The patient, a 53-year-old male, underwent amputation and an osteotomy at the tibial diaphysis.The fluorescence intensity of the two images is color-coded in green (150 s) and purple (250 s) for comparison before [Fig.4(a)] and after correction with fixed-reference [Fig.4(b)] or frame-by-frame reference [Fig.4(c)].As indicated in the corresponding zoom-in images of the yellow-box areas in Figs.4(a)-4(c), the initial mismatch of ∼2.5 mm in the boundaries of the bone and soft tissue is observed before the correction [Fig.4(d)].After the fixedreference correction, the images were overcorrected with a 2 mm mismatch [Fig.4(e)].However, the frame-by-frame correction significantly improved the alignment of the two images [Fig.4(f)].

Fig. 3
Fig. 3 Phantom study results demonstrating motion artifact reduction from mechanical disturbances and simulated respiration.(a) and (b) The average intensity fluctuations within the ROI for the 300 and 100 nM wells, respectively, before (blue) and after (red) applying the motion correction.(c) and (d) The changes in average intensity due to simulated respiratory artifacts within the ROI for the 300 and 100 nM wells, respectively, also before (blue) and after (red) correction.

Fig. 4
Fig. 4 An example of applying MI-based motion correction to a patient's DCE-FI.(a)-(c) The overlay of two frames captured at 150 and 250 s after the start of imaging.The green and purple colors represent the images captured at 150 and 250 s, respectively.(a) Before correction, (b) after fixedreference correction, and (c) after frame-by-frame correction.(d)-(e) Zoom-in images of the yellow dashed-line areas indicated in panels (a)-(c), respectively.

Fig. 5
Fig. 5 Joint histograms of the two frames in Fig. 4: (a) before correction, (b) after fixed-reference correction, and (c) frame-by-frame correction.

Fig. 6
Fig. 6 MI-based motion correction improved the kinetic curves: (a) white-light image of the surgical view; (b) with fluorescence image overlaid and four circular ROIs drawn, two on the proximal region (p1, p2) and two on the distal region (d1, d2); and (c) kinetic curves of the four ROIs in corresponding colors, where dashed and solid lines represent kinetic curves before and after the correction, respectively.

Fig. 7
Fig. 7 MI-based motion correction improved the perfusion-related parameters before and after the correction, in each of the proximal and distal circular ROI shown in Fig. 6(b): (a) average I max , (b) average IS, and (c) average TTP.

Fig. 8
Fig. 8 The total corrected displacements of each imaging session in (a) x -and (b) y -axes.

Fig. 9
Fig. 9 Comparative analysis of ICG intensity in the tibial bone region of 15 patients' postlower extremity amputation against phantom wells with ICG-equivalent concentrations.(a) Average histogram distributions from multiple imaging sessions display baseline (top), postosteotomy (middle), and postcircumferential periosteal/soft tissue stripping (bottom) measurements, with the 90% confidence intervals shaded.Diagonal lines highlight the area within the mFWHM of the average histograms.(b) Relationship between fluorescence intensity and ICG-equivalent concentrations for the validation phantom.The shaded region delineates the range of ICG intensity corresponding to mFWHM of the average histograms and associated concentrations.