28 June 2016 Three-dimensional registration of intravascular optical coherence tomography and cryo-image volumes for microscopic-resolution validation
Author Affiliations +
J. of Medical Imaging, 3(2), 026004 (2016). doi:10.1117/1.JMI.3.2.026004
Evidence suggests high-resolution, high-contrast, 100  frames/s intravascular optical coherence tomography (IVOCT) can distinguish plaque types, but further validation is needed, especially for automated plaque characterization. We developed experimental and three-dimensional (3-D) registration methods to provide validation of IVOCT pullback volumes using microscopic, color, and fluorescent cryo-image volumes with optional registered cryo-histology. A specialized registration method matched IVOCT pullback images acquired in the catheter reference frame to a true 3-D cryo-image volume. Briefly, an 11-parameter registration model including a polynomial virtual catheter was initialized within the cryo-image volume, and perpendicular images were extracted, mimicking IVOCT image acquisition. Virtual catheter parameters were optimized to maximize cryo and IVOCT lumen overlap. Multiple assessments suggested that the registration error was better than the 200-μm spacing between IVOCT image frames. Tests on a digital synthetic phantom gave a registration error of only +1.3±2.7  μm (signed distance). Visual assessment of randomly presented nearby frames suggested registration accuracy within 1 IVOCT frame interval (−25.0±174.3  μm). This would eliminate potential misinterpretations confronted by the typical histological approaches to validation, with estimated 1-mm errors. The method can be used to create annotated datasets and automated plaque classification methods and can be extended to other intravascular imaging modalities.
Prabhu, Mehanna, Gargesha, Brandt, Wen, van Ditzhuijzen, Chamie, Yamamoto, Fujino, Alian, Patel, Costa, Bezerra, and Wilson: Three-dimensional registration of intravascular optical coherence tomography and cryo-image volumes for microscopic-resolution validation



Intravascular optical coherence tomography (IVOCT) has demonstrated great promise in identifying key plaque features due to its resolution (10 to 20  μm), noise, and contrast among rival intravascular imaging modalities (intravascular ultrasound, near infrared spectroscopy, and intravascular MRI). IVOCT has been used to differentiate lipid, calcium, and fibrous tissue1,2 and can quantify macrophage content within atherosclerotic plaques.6 Furthermore, it is the only modality with the ability to quantify fibrous cap thickness: a powerful indicator of the likelihood of thin cap fibroatheroma (TCFA) rupture,78.9 which is believed to be the most frequent cause of myocardial infarction.1011.12.13

However, challenges exist. Identification of a TCFA is not always straightforward. In particular, it is sometimes difficult to distinguish between lipid pools and large calcium deposits.3,1415.16 Additionally, superficial attenuation due to the presence of macrophages near the vessel intima, and signal attenuation resulting from oblique incidence of the IVOCT imaging beam, may also lead to misclassification of a region as TCFA.17 Quantitative methods have been employed to automatically classify IVOCT images,1819.20.21 but these are limited in scope, as many are two-dimensional (2-D), and many have insufficient data for machine learning approaches, which typically require a significant amount of data to cover the broad spectrum of vessel and lesion types an analyst may encounter. These challenges suggest a need for further validation studies of IVOCT imaging.

Many IVOCT validation studies currently use paraffin histology as the “gold standard” for validation,2,18 but challenges exist with using this approach. In one typical study,18 interesting locations were identified along the vessel, marked with insertion of a needle visible in IVOCT, and imaged with a stationary IVOCT acquisition. Needles were then replaced with sutures, and tissue was fixed. Next, sutures were removed and replaced with ink marks that would be visible in histology. Finally, histological sections were obtained and compared to the stationary IVOCT images. In such an approach, several aspects can limit the accuracy of spatial correspondence between histology and IVOCT. First, sutures and ink marks tend to be large compared to IVOCT sampling (100 or 200  μm). Second, histological sections are often spatially distorted (tearing, folding, shearing, and shrinking), limiting the ability to make correlations even if the samples are in the same z location.22,23 Third, there is no accounting for any oblique orientation of the catheter with respect to the vessel wall. Fourth, if one uses pullback acquisitions mimicking clinical imaging, it is even more difficult to ensure spatial correspondence between an IVOCT image and histology. Fifth, using these methods, it would be very difficult to try to relate selected 2-D histology images to the up to 500 images in a three-dimensional (3-D) IVOCT pullback.

Cryo-imaging is an imaging tool that should help fill the gap between 3-D IVOCT pullbacks and 2-D histology. Briefly, the system (CryoViz™, BioInVision) consists of a cryo-microtome with an integrated imaging system.24,25 The system serially sections and acquires bright-field and fluorescence microscopic image volumes. Visualization and analysis software is used to generate high-resolution color and fluorescence volume renderings of vessels in which plaque architecture and components are fully preserved. This provides an accurate depiction of the vessel geometry without the distortion and artifacts resulting from standard histological fixation and processing.26,27 This also enables precise localization of individual slices in the context of the vessel. Furthermore, the system allows for individual sections to be collected and processed for standard histological staining. Because the histological sections are acquired with a tape system ensuring geometric fidelity, and because the cryo-images are obtained on precisely the same tissue, exact registration to the block-face image is possible.

Several studies have been performed by our group to show the ability to use cryo-imaging as ground truth as validation for plaque characterization. Nguyen et al.27 first demonstrated the ability to manually classify 15 histologically distinct vascular features. In Nguyen et al.’s study, blinded readers identified 15 tissue types, but classification improved when data were collapsed to 11 tissue types. If we further collapse data (e.g., both light and dense calcifications are labeled calcifications) to give lipid, fibrous, calcium, and other, we get excellent sensitivity/specificity for the three plaque types: 86/99, 83/99, and 92/98, respectively. This result is important, as this was a large study with 344 tissue samples, each verified by histology. Mehanna et al.28 used cryo-images to validate the ability of IVOCT to perform volumetric coronary calcium assessment. Faramarzalian et al.29 demonstrated the ability of cryo-images to aid in assessment of inflammation in coronary plaque lesions in IVOCT studies.

In order to use cryo-imaging to routinely provide validation data for IVOCT images, we developed a specialized registration algorithm to match volumes of IVOCT images with corresponding cryo-image volumes. The major obstacle is that the cryo-image volume is a true 3-D volume, while the IVOCT volume is obtained from a pullback with the catheter tip as the reference for each image. We develop the algorithm and validated it on a synthetic digital phantom, as well as actual IVOCT and cryo-image data. We then demonstrate how our work can be used to validate IVOCT data without the potential mischaracterization from conventional histology techniques. Finally, we show how this work can be applied toward developing automated IVOCT plaque characterization software.


Image Processing: Registration of Cryo-Image and Intravascular Optical Coherence Tomography Pullback Volumes

3-D image registration software was created to accurately match a given IVOCT image with an appropriate image from the cryo-image volume (Fig. 1). The challenge is that IVOCT images are obtained perpendicular to the imaging catheter with the imaging catheter always at the center of the image, while the cryo-image volume faithfully reflects the true 3-D geometry and vessel shape at a much higher sampling rate.

Fig. 1

Registration of IVOCT pullbacks with a catheter-based reference with true 3-D cryo-image volumes. The video details the steps involved in our IVOCT–cryo registration algorithm. (1) We select a vessel segment with visible side branch fiducials present in both the IVOCT- and cryo-image volumes. (2) We then segment the lumen in both image volumes. (3) We insert a virtual catheter into the cryo lumen volume and extract perpendicular images, digitally mimicking IVOCT image acquisition within the cryo-image volume. (4 and 5) We optimize parameters of the virtual catheter according to a cost function that maximizes lumen overlap between the corresponding pairs of images, as determined by cross-correlation. (6) The result of optimization is a registered cryo-lumen volume that matches the IVOCT lumen volume. (7) The coordinates of the registered cryo-lumen volume are then applied to the original color cryo-data to obtain a registered color cryo-image volume having image-to-image correspondence with its matching IVOCT volume. (Video 1, mp4, 4233 KB) [URL:  http://dx.doi.org/10.1117/1.JMI.3.2.026004.1].


Our solution was to digitally mimic the IVOCT acquisition process within the cryo-image volume. We created a virtual catheter within the cryo-image volume and extracted perpendicular reformatted images. Since lumen segmentation is accurate in both modalities, we created an objective function based on the overlap of segmented binary images and used normalized cross-correlation to compute cost. This first step involved selecting volumes for registration from both the cryo- and IVOCT images. During IVOCT imaging, this volume was marked with easily visible sutures at locations pertaining to side branches. Cryo-imaging was then performed at the locations marked with sutures. Since the most consistent feature in both IVOCT and cryo-imaging is the vessel lumen, we perform registration on the segmented lumen volume from both modalities. In the end, the registration parameters are applied to the full color cryo-data. The output was a volume of cryo-images that accurately matched the corresponding IVOCT images acquired from the same vessel.

We modeled the virtual catheter as a second-order polynomial, governed by the equations below.

Virtual catheter model:






where [A,B,C,D,E,F] are coefficients of the polynomial terms, [k0] is the starting index in the dataset, [kc] is the current index in the dataset, [αR] is the scaling factor along the radial direction, [αZ] is the scaling factor along the z direction, and [Δzcryo] is the z resolution of the cryo-image volume.

To register the cryo and IVOCT volumes, we interactively select a point along the virtual catheter that corresponds to the first IVOCT image in a volume, using large side branches as fiducials. We then calculate the normal to the catheter at that point using the equations below.

Normal to virtual catheter:






Using the point P0=(Xc,Yc,Zc) and normal N=Xc,Yc,Zc, we can compute a plane within the cryo-volume, which corresponds to a particular IVOCT image. The equation for the plane is given as


where [x,y,z] is the position of each pixel in the IVOCT image plane. The pixel spacing of the plane is determined by the in-plane IVOCT resolution ΔrOCT.

The plane is then rotated by an angle [θ] to account for the rotation that takes place within the IVOCT dataset.30,31 Each subsequent slice is then adjusted according to [dθ/dz], which is the change in rotation that takes place as you move along the virtual catheter.

We then traverse a distance along the virtual catheter as determined by the z resolution of the IVOCT pullback ΔzOCT to find the starting point of the next image.


This process is repeated until each IVOCT image is matched with a corresponding cryo-image.

The catheter model was interactively initiated and iteratively refined using Nelder–Mead simplex optimization. Interactive initiation was done using a MATLAB®-based (Mathworks, Natick, Massachusetts) computer interface, where parameters were manually adjusted by a user until a suitable starting condition was achieved. The interface allowed users to visually see how well each frame of the datasets overlapped for a given set of initial parameters. In total, each dataset involved optimizing the following 11 parameters in order to find the best overlap between the IVOCT and cryo-images: Φ=[A,B,C,D,E,F,k0,αR,αZ,θ,dθ/dz]. Pseudocode for the algorithm is shown in Table 1.

Table 1

The pseudocode for our algorithm, as displayed in Fig. 1.

Pseudocode for Registration of IVOCT and Cryo-Image Volumes
1. Preprocess:
   a. Deteremine volumes for registration in both cryo- and IVOCT image datasets
   b. Segment the lumens of both volumes using interactive segmentation
2. Initialize:
   a. Interactively initialize Φ=[A,B,C,D,E,F,k0,αR,αZ,θ,dθ/dz]
3. OptimizeΦ using Nelder–Mead Simplex Optimization
Objective Function {
For each image pair
      1. Determine proper point along virtual catheter model at which to interpolate cryo-image
      2. Determine normal vector to the virtual catheter at determined point
      3. Find tangent plane to virtual catheter using determined point and normal vector
      4. Determine proper rotation of the tangent plane
      5. Interpolate cryo-image in the tangent plane using nearest neighbor interpolation
      6. Determine normalized cross-correlation between IVOCT image and interpolated cryo-image
      7. Find position along catheter for next image
   Average normalized cross-correlation across all image pairs




Specimens and Intravascular Optical Coherence Tomography Imaging

We obtained coronary arteries from the Cuyahoga County Coroner’s office from human cadavers within 72 h of death. Arteries were removed and stored at 4°C after removal from the body and prior to IVOCT imaging. The time between harvest and IVOCT imaging did not exceed 1 week. This procedure was deemed in accordance with federal, state, and local laws by the Case Institutional Review board.

Preparation of each artery was done prior to IVOCT imaging. A luer was sutured to the proximal end of each vessel. Vessels were flushed with saline to remove blood from the lumen. Major side branches and the distal end of each artery were sutured shut. Using super glue, the artery was adhered to the sides and bottom of a rig that was used to minimize motion between imaging procedures. The rig consisted of two hollow plastic cylinders that were cut in half and overlapped to match the tissue size.

IVOCT imaging is now described. IVOCT image acquisition was performed using a state-of-the-art IVOCT imaging system, LightLab’s C7-XR Fourier domain IVOCT intravascular imaging system (C7-XR, LightLab Imaging Inc., Massachusetts). The recently FDA-approved C7-XR is about 10× faster than the previous generation of time domain IVOCT. This provides a high density of A-lines per frame, which ultimately improves tissue features and consequently plaque characterization. The high sampling rate also enables faster pullbacks, minimizing motion artifacts favoring 3-D reconstructions of the vessel. Pullbacks were performed at 10.0 or 20.0  mm/s using 50,000 A-Lines/s yielding 5.4 cm pullbacks at 100 or 200  μm frame intervals. Vessels were pressurized with saline or optimal cutting temperature compound (Tissue Tek, Ted Pella, Inc., Redding, California) during the pullback to prevent vessel collapse. After IVOCT imaging, IVOCT images were reviewed and sutures were placed on the vessel corresponding to the start and stop points of regions of interest (ROIs) along the vessel. Then, the imaging rig was filled with optimal cutting temperature compound. Following this, the entire rig was covered in aluminum foil and snap frozen by placing it in a container filled with liquid nitrogen. The frozen specimen was then stored in a 80°C freezer until cryo-imaging was performed.


Cryo-Imaging and Acquisition of Corresponding Histology

Preparation of each artery was done prior to cryo-imaging. Arteries were removed from the freezer and cut into blocks corresponding to the ROIs determined during IVOCT imaging. Prior to imaging, each block was placed in the cryo-imaging system to allow equilibration to the 20°C cutting temperature.

Cryo-imaging is now described. The cryo-imaging system consists of a modified large section cryo-microtome (8250 large section cryostat, Vibratome, St. Louis, Missouri), XYZ robotic positioner carrying an imaging system that consists of a stereo microscope (SZX12, Olympus, Japan), GFP fluorescent filters (exciter: HQ470/40x, dichroic: Q495LP, emitter: HQ500LP, Chroma, Rockingham, Vermont), low light digital camera (Retiga Exi, QImaging., Canada), and color light source (XCite 120PC, EXFO, Canada). The cryo-imaging system is controlled by a control computer running Labview (National Instruments, Austin, Texas). The stereomicroscope used an objective of 0.11 NA and a zoom settings of 7 to 16×. Once equilibrated to cutting temperature, blocks were fixed to the stage of the cryo-microtome using optimal cutting temperature compound. The specimen was then sectioned until the surface of the block face was flat. The bright-field lamp was turned on and the camera was positioned over and focused onto the region of the block-face surface containing the vessel. A white board was placed over the surface of the block face and was used to white-balance the camera. The white board was removed and the bright-field exposure time for the camera was determined. For color imaging, typical exposure times for the green channel were 10  ms. The color lamp was turned off, the fluorescent light was turned on, and the fluorescent exposure time was determined. For fluorescent imaging, typical exposure times for the green channel were 3000  ms. The machine was then set to run such that a 20-μm section was removed from the specimen, a bright-field followed by a fluorescence image of the block face was acquired, and the process was repeated until the whole specimen was imaged.

Histology was acquired at selected sections using the Kawamoto method.32 Briefly, a cryofilm was applied to the specimen surface prior to sectioning. As the specimen was sectioned, the tissue section was left attached to the cryofilm. Histological staining was then performed on the cryofilm before it was finally mounted onto a standard 1×3 glass slide. Histology slices were then digitized and registered to their corresponding cryo-images using 2-D affine registration with scale, rotation, and translation as free parameters.


Image Processing Software

Several preprocessing steps are required on the cryo-image data, some of which are now standardized on our cryo-imaging system. The steps involve using Amira (Mercury Computer Systems Inc., Chelmsford, Massachusetts) and software written in MATLAB® (Mathworks, Natick, Massachusetts).

First, cryo-images are corrected for nonuniform illumination of the block face during imaging. A reference image of a white board is obtained prior to imaging. This image is scaled from 0 to 1 and each bright-field cryo-image is divided by the scaled reference image. Next, cryo-images are corrected for small slice-to-slice misalignments using grayscale registration. Briefly, we convert color cryo-images to grayscale and optimize the normalized cross-correlation as a function of x and y subpixel shifts using a fast heuristic search. Because of the geometry of the imaging system, rotation is not required. The same transformation parameters are applied to both the bright-field and fluorescence cryo-volumes. Finally, fluorescence cryo-images are corrected for subsurface fluorescence using “next-image” processing as described by Steyer et al.24 Briefly, each image is attenuated, blurred, and subtracted from the image immediately preceding it in the image volume. Thus, highly fluorescent regions that are actually beneath the region being imaged, yet emit light to the surface of the block face during acquisition, are corrected for.

Registration software was implemented in MATLAB® (Mathworks, Natick, Massachusetts). Initiation was done interactively by manually tweaking individual parameters until a suitable overlap between all lumen pairs was determined by visual assessment. The parameters were then iteratively refined using Nelder–Mead simplex optimization in MATLAB®, where cross-correlation between lumen pairs was used as the cost function.

Registration of histology to corresponding cryo-images was done by digitizing histology slides using an Olympus VS120 microscope. Digitized slides were then registered to their corresponding cryo-images using an affine transformation with scale, rotation, and translation as free parameters. Mutual information was used as the cost function for cryo- to histology registration.

All image processing software was implemented on a Dell Precision T7610 using a 64-bit edition of Windows 7 Professional, with two 2.60 GHz Intel® Xeon® CPU and 128 GB of RAM.


Assessment of Registration Accuracy

We developed a digital phantom to validate the accuracy of our registration algorithm. To create the phantom, we used a binary lumen volume from a cryo-image dataset as a reference. We generated a sequence of 10 images from this volume using a known set of parameters with our virtual catheter model. These images were designated as “synthetic IVOCT images.” Sets of initial parameters were chosen to include extremes of initiation errors that we thought reasonable and mixtures of these errors. Additionally, we analyzed the effect of modifying the two stopping criteria: minimum change in parameter step size and minimum change in the cost function. These stopping criteria were modified and their impact on registration accuracy was assessed. Because the images were digitally created, we knew the exact position of each voxel in the phantom images. Having this information, we were able to determine the Euclidean distance between the reference and our determined solution as our measure of registration quality.

To determine registration quality on actual data, we created visual overlays of registered image pairs and assessed continuity of features. Additionally, we conducted an experiment where we showed two expert IVOCT image analysts an IVOCT image and five corresponding cryo-images, which contained easily identifiable features, such as a side branch or well-defined calcified lesions. The five images contained the registered image as determined by our algorithm, two images before the registered image, and two images after the registered image. The images were spaced according to the frame interval of the IVOCT image (200  μm). The analysts then selected the image that they felt matched best. Each analyst was shown 20 such cases.

Additionally, we quantitatively assessed registration between IVOCT and cryo-image volumes by analyzing lumen area between registered images, as well as the DICE coefficient of the corresponding lumens.


Annotation of Intravascular Optical Coherence Tomography Images for Classifier Training

We created a methodology for annotating IVOCT images to provide ground truth for future plaque classification studies. Using the registered cryo-images as a reference, an expert analyst marked every pixel in the IVOCT images that contained visible regions of fibrous, lipid, or calcium plaque. This database of annotated IVOCT images can be used in future studies to validate manual and automated approaches that aim to identify these plaque types.




Cryo-Imaging/Histology Identification of Major Plaque Types

Cryo-imaging allows easy identification of common plaque types confronted in IVOCT imaging. In color and fluorescent cryo-images, one can determine features associated with normal, lipid, calcified, and fibrous plaque types, as shown in Fig. 2. In the normal example (row 1), we observe three concentric vessel layers that are red to pink hued in the color image and dimly green in the fluorescent image, with the medial layer being dimmest, the adventitial layer being next in brightness, and the intimal layer being mildly fluorescent, which is indicative of early intimal thickening.11 Calcified lesions (row 2) are characterized by a chalky white appearance in the color image and high green signal in the fluorescent image. Lipid lesions (row 3) are characterized by a yellow appearance in the color image and a dim appearance in the fluorescence image. Fibrous tissue is seen (rows 2 and 3) in areas where the intimal layer of the vessel is inflamed and is characterized by a pink appearance in the color image and a medium to bright appearance in the fluorescent image. Although there is some biological variability, these general observations hold across samples, as was shown previously using a large number of cryo-images with histological verification.27

Fig. 2

Major plaque tissue types are recognizable in cryo-images. Cryo-imaging easily distinguishes the major plaque types (calcium, fibrous, and lipid) confronted during IVOCT imaging. Normal vessels (row 1) are characterized by three concentric layers with a pink-red appearance in the color image and a dim to mildly bright appearance in the fluorescent image. Calcified lesions (row 2) are characterized by a chalky white appearance in the color image and high green signal in the fluorescent image. Lipid lesions (row 3) are characterized by a yellow appearance in the color image and a dim appearance in the fluorescence image. Fibrous tissue is seen (rows 2 and 3) in areas where the intimal layer of the vessel is inflamed, and is characterized by a pink appearance in the color image and a medium to bright appearance in the fluorescent image. There are large regions of fibrous tissue, and F identifies a classic appearance.


These findings from cryo-imaging can be further confirmed by cryo-histology, which can be acquired within our imaging workflow (Fig. 3). Simple affine registration matched the cryo-image with its corresponding histology with very high accuracy (<10  μm), as evidenced by the near exact overlap of microscopic vessel features.

Fig. 3

Plaque characterization by cryo-imaging can be validated by histology. Histology can be acquired during the cryo-imaging procedure without disrupting the normal image acquisition process. H&E is shown on the left row, and Masson’s trichrome is shown on the right. Exact correspondence between vascular features is shown when cryo color, fluorescent, and histology images are overlaid. In the calcium overlay, near exact continuity of the calcium border is seen across all images. In the fibrous overlay, exact continuity is seen in collagen strands in the fibrous lesion, as well as in the luminal and adventitial borders.



Assessing Registration Quality with a Digital Synthetic Phantom

The digital phantom experiment assessed the registration accuracy of our algorithm (Sec. 3.4). We evaluated the effect of potential local minima by starting the algorithm with different initial parameters and recording the final registration results. Initial parameters were varied over ranges that would exceed the error in manually determined starting parameters. At these initial conditions, image transformations were clearly in error, with image volume overlap as small as 10%. Over 24 cases, the mean voxel error was +1.3±2.7  μm, as determined by the signed Euclidean distance along z, with the minimum (maximum) error being +0.0±0.0  μm (+7.2±3.6  μm). The method was quite robust to the starting condition. With regards to stopping criteria, we obtained excellent results when either the change in parameter step size or the change in the cost function was <106. The parameter step size is relative to the normalized value of 1.0 described previously. In practice, in nearly all cases, optimization stopped due to the parameter step size criterion. It should be noted that parameter scaling was crucial to get good results in the Nelder–Mead simplex.33

Additionally, we assessed susceptibility to local minima when registering actual data. We ran over 20 cases of repeated registrations and found that variations were typically within <100  μm of each other.


Assessing Registration Accuracy of Intravascular Optical Coherence Tomography to Cryo-Volumes

We assessed the quality of registration for actual IVOCT and cryo-image volumes. We used a sector display to visualize the quality of registration (Fig. 4). There was excellent continuity of features between the cryo- and IVOCT images, especially in the luminal and adventitial boundaries.

Fig. 4

Image overlays of registered IVOCT and cryo-image pairs suggest very high registration accuracy. To assess registration accuracy, we overlayed IVOCT and registered cryo-image pairs and found very high correspondence. A representative example is shown above. (a) IVOCT image and (c) registered cryo-image. (b) When we overlay the images, we can see very high continuity between vascular features. This is especially apparent at the inner (lumen) and outer (adventia) boundaries of the vessel. Slight differences are likely due to physical deformation of the vessel that took place between IVOCT and cryo-imaging procedures.


Additionally, in a sequence of registered images, we compared the frames before and after the corresponding image (Fig. 5). We found that each frame matched its corresponding image better than the frame before and the frame after. This suggests that the accuracy of our registration is within 1 frame interval (±200  μm).

Fig. 5

Qualitative analysis of registered cryo- and IVOCT image datasets suggests z-registration accuracy is within 1 IVOCT frame interval (±200  μm). We visually compared registered cryo- and IVOCT image pairs, as well as images located immediately before and after within the registered datasets. Qualitative features suggested that registration accuracy was within 1 frame interval. In the above case, this is especially apparent when considering the side branch located within the registered pair. In the bottom cryo-image, located +200  μm away from the registered pair, we see that the side branch has yet to appear, as it has in the registered pair. Moreover, in the top cryo-image, located 200  μm away from the registered pair, we see that the side branch is more pronounced than in the registered pair. Clearly, the best match occurs within the registered pair, suggesting that registration accuracy is within the IVOCT frame interval of ±200  μm.


To further support this claim, we performed a quantitative assessment with two IVOCT image analyst experts (Fig. 6) as described in Sec. 4.3. We compared the two analysts’ choices of best match with our algorithm’s best match. The results are summarized in the histogram shown in Fig. 6. We computed the error (mean±SD) from both analysts to be 25.0±174.3  μm (0.1±0.9  frames), further suggesting a registration accuracy within 1 frame interval. Note that the mean is accurate with very little bias and that the actual precision might be better since our smallest measurement interval is 1 frame (200  μm).

Fig. 6

Assessment by IVOCT analysts further supports the notion that registration in within 1 frame. To further support the claim that cryo-image registration accuracy is within the frame interval of IVOCT image acquisition (±200  μm), we showed 20 representative cases (two per vessel) from all registered datasets to two analysts. The analyst was shown an IVOCT image as well as five consecutive cryo-images. The cryo-images contained the image that our algorithm said best matched the IVOCT image, as well as two images located immediately before (200, 400  μm) and two images immediately after (+200, +400  μm) within the data set. The analyst then picked the one that best matched the IVOCT image. The histogram shows the result of the experiment and suggests registration accuracy is within ±1 frame interval of the IVOCT image.


We applied additional quantitative metrics to assess registration of IVOCT and cryo-image data. We compare lumen areas and DICE coefficients for corresponding frames in each registered volume. A clear correlation is observed in both metrics. Bland–Altman analysis comparing cryo and IVOCT lumen area showed 401 of 421 cases within the 95% confidence interval, with mean and standard deviation of the differences being 0.01 and 0.43  mm2, respectively. Mean DICE coefficients [Fig. 7(b)] for all cases were 0.91±0.04.

Fig. 7

Qualitative lumen metrics again suggest very high registration between cryo- and IVOCT image pairs. We looked at the lumen area and DICE metrics comparing registered cryo- and IVOCT image pairs as a quantitative metric for assessing registration accuracy. A comparison of (a) lumen area and (b) DICE for all cases is shown, suggesting very good agreement in lumen area and overlap. In (a), Bland–Altman analysis compares the lumen areas for all 441 registered pairs generated in the study. We see 421 of 441 cases within the 95% confidence interval of agreement, with mean and standard deviation of the differences being 0.01 and 0.43  mm2. Mean DICE for each case is shown in (b), with standard deviation shown in the error bars. Mean DICE for all cases were 0.91±0.04. Both metrics suggest very high registration accuracy.



Application to Plaque Characterization

To highlight the importance of accurate registration for plaque characterization and analysis, we show a registered case containing three sequential IVOCT and cryo-image pairs, each spaced 1 mm apart (Fig. 8). The top row shows a lipid plaque, the middle row shows the inception of a lipid plaque, and the bottom row shows a fibrous lesion. A registration error of 1 mm, a value presumed by existing validation techniques used in the literature,2,18 could lead to vastly different conclusions regarding plaque characterization and quantification. We see for the baseline IVOCT case that a registration error of only 1 mm in the validation modality could preclude the identification of lipid plaque. Morphological assessments would be compromised as well. Clearly, accurate registration in validation is necessary for making correct assessments of IVOCT images.

Fig. 8

Accurate registration in validation modality is necessary for accurate plaque characterization and assessment in IVOCT images. Above, we see a registered case showing three sequential images spaced 1 mm apart. The top row shows a lipid plaque (L), the middle row shows the inception of a lipid plaque (L) within a predominantly fibrous lesion (F), and the bottom row shows an entirely fibrous lesion (F). We see for the baseline case, a misregistration of 1 mm in the validation modality could preclude the identification of lipid. Morphological assessments would be compromised as well.


Our work can be used to provide annotated IVOCT image datasets for manual and automated plaque characterization (Fig. 9). Although registration is quite accurate, we choose to use cryo-images as a guide to annotate IVOCT images. Due to potential small registration errors, this is more accurate than annotating the cryo-images and copying the results directly to IVOCT. In Fig. 9, fibrous, calcium, and lipid regions are annotated in the IVOCT images using the registered cryo-images as a guide.

Fig. 9

Registered cryo-images can be used to train and test automated plaque characterization methods for IVOCT images. We generated 441 registered image pairs from 10 vessels. Using the registered cryo-image as reference, we can annotate every pixel in an IVOCT image corresponding to a major plaque type (fibrous, lipid, and calcium). Using these annotated images, we can test and train a classifier to automatically characterize IVOCT images. (a–c) A case containing a calcified lesion, denoted by the letter ‘C’. (d–f) A case containing a lipid lesion, denoted by the letter “L.” Fibrous tissue is shown as a gray ring in both panels C and F.





Three-Dimensional Validation of Intravascular Optical Coherence Tomography Images Achieved with High Registration Accuracy

In this study, we developed a methodology to provide accurate 3-D validation of IVOCT pullback volumes using microscopic, color, and fluorescent cryo-image volumes, with optional exactly registered cryo-histology. We show examples of clear identification of plaque types using cryo-imaging, with very accurate registration to individual IVOCT frames, enabling one to find a feature in cryo-images and look for the corresponding response in IVOCT. Potential applications include at least verification of IVOCT manual classification, investigations of plaque pathobiology, creation of an IVOCT plaque atlas or physician training data, and creation of databases for machine learning plaque classification algorithms. The method is amenable for application to other vascular imaging modalities, including other intravascular methods (IVUS, intravascular fluorescence, and intravascular MRI).

Like previous reports,26,27 we show some examples of using cryo-imaging to clearly identify major plaque components (fibrous, calcium, and lipid) and normal vascular tissue. Results here are consistent with those from a previous report with 344 cryo-images, each with histological backup.27

We had a positive experience with the tape-based cryo-histology technique. We demonstrated the use of two types of stains: H&E and Masson’s Trichrome, a stain commonly used in fibrous cap studies to assess collagen cap thickness. The tape acquisition method enabled acquisition of spatially accurate sections that could be exactly registered to the cryo-volume, as evidenced by exact continuity between vascular features in the registered cryo- and histology images (Fig. 4). In practice, we find that it is robust, fast, and easy to acquire and process H&E histology. The entire process requires ∼30 s to acquire a section on tape and 3 to 4 min to stain and mount, making it possible to regularly perform H&E histology. Processing with Masson’s Trichrome is more involved and less robust.

For cryo–IVOCT registration, we created a specialized image registration method. The issue is that IVOCT is obtained with the catheter tip as the image reference, while cryo-imaging reveals the true shape of the vessel. Our solution consisted of creating a virtual catheter path through the cryo-image volume. At least over the segment lengths studied here, a second-order polynomial was sufficient for modeling the path. We experimented with a grayscale objective function before arriving at the lumen volume overlap. The problem with using grayscale values is that the images were quite different, especially considering that IVOCT has shadows and attenuation to baseline with increasing depth in the tissue, reducing correspondence, even when one uses mutual information as the objective function. Rather than try to correct grayscales, we opted for binary lumen volume overlap. We found that there were sufficient features (side branches and nonconcentric lumens) that lumen volume worked well. It is understood that we could use lumen surface-to-surface registration, probably achieving faster registrations. Nevertheless, we chose to continue with lumen volume overlap because of simplicity in coding and because it would be easy to include additional features such as a segmented calcification if need be.

Registration accuracy is better than the spacing between IVOCT image frames (200  μm). Errors estimated with the digital synthetic phantom were very small (+1.3±2.7  μm). When operators evaluated registration accuracy on real data (Sec. 3.4), they computed an error with mean±SD of 25.0±174.3  μm (0.1±0.9  frames). It is quite possible that this experiment overestimated errors due to the 200  μm spacing between considered images. DICE and lumen area metrics [Figs. 7(a) and 7(b)] matched very well. Visually assessed sector displays showed good registration (Fig. 5). Small displacement errors seen in some sector displays were likely due to small distortions of the vessels between IVOCT imaging and cryo-imaging. Rather than continuing the registration process, possibly with additional deformable registration, we chose another strategy. We go to great lengths to register the appropriate cryo-image to the corresponding IVOCT image. We then allow the operator to annotate the IVOCT image using the registered cryo-image as a guide. There is little or no ambiguity in annotation. Registration error on the order of 50 to 100  μm is far less than the 1-mm estimated error reported for other methods.2,18

Our proposed method should limit the misinterpretations that are clearly possible with other methods (Fig. 8). As described in Sec. 1, previous methods using ink marks will have limited accuracy. Our study confirms that the ink-mark method may not be sufficiently accurate for validating smaller features, such as the fibrous cap, as reported previously.2 In Fig. 8, plaque type and morphology can change drastically even over a distance as short as 1 mm. Furthermore, the added benefit of having 3-D data allows one to analyze adjacent regions to individual IVOCT frames as well, effectively allowing one to circumvent the effects any registration error may have on plaque characterization.

We believe that our method will be ideal for creating validation data for automated classification (Fig. 9). Each tissue voxel in an IVOCT volume can be labeled, allowing even voxel-based classification rather than pixel-based classification. This would allow one to utilize 3-D information in classification, which would not be possible using isolated histology images for validation.

Finally, our methods can be used to validate other intravascular imaging modalities as well, including intravascular MRI, intravascular ultrasound, near-infrared fluorescence, and multimodality solutions that combine these modalities. Each of these methods employs a catheter pullback in image acquisition, which can be handled using our methodology.



We believe that cryo-imaging is a promising technology for validation of 3-D IVOCT imaging studies. It enables heretofore unavailable registration accuracy between IVOCT imaging data and 3-D microscopic cryo-image data with optional histology. The method will provide 3-D data for training of IVOCT plaque classification algorithms. The virtual catheter registration method and cryo-imaging should be applicable for assessments of other intravascular imaging modalities.


This project was supported by the National Heart, Lung, and Blood Institute through grants NIH R21HL108263 and 1R01HL114406-01, the Choose Ohio First Scholarship, and by the National Center for Research Resources and the National Center for Advancing Translational Sciences through Grant No. UL1RR024989. These grants were attained via collaboration between Case Western Reserve University and University Hospitals of Cleveland.


1. I. K. Jang et al., “Visualization of coronary atherosclerotic plaques in patients using optical coherence tomography: comparison with intravascular ultrasound,” J. Am. Coll. Cardiol. 39(4), 604–609 (2002).JACCDI0735-1097 http://dx.doi.org/10.1016/S0735-1097(01)01799-5 Google Scholar

2. H. Yabushita et al., “Characterization of human atherosclerosis by optical coherence tomography,” Circulation 106(13), 1640–1645 (2002).CIRCAZ0009-7322 http://dx.doi.org/10.1161/01.CIR.0000029927.92825.F6 Google Scholar

3. W. M. Suh et al., “Intravascular detection of the vulnerable plaque,” Circ.: Cardiovasc. Imaging 4(2), 169–178 (2011). http://dx.doi.org/10.1161/CIRCIMAGING.110.958777 Google Scholar

4. R. Puri, M. I. Worthley and S. J. Nicholls, “Intravascular imaging of vulnerable coronary plaque: current and future concepts,” Nat. Rev. Cardiol. 8(3), 131–139 (2011). http://dx.doi.org/10.1038/nrcardio.2010.210 Google Scholar

5. H. G. Bezerra et al., “Intracoronary optical coherence tomography: a comprehensive review: clinical and research applications,” JACC: Cardiovasc. Interv. 2(11), 1035–1046 (2009). http://dx.doi.org/10.1016/j.jcin.2009.06.019 Google Scholar

6. G. J. Tearney et al., “Quantification of macrophage content in atherosclerotic plaques by optical coherence tomography,” Circulation 107(1), 113–119 (2003).CIRCAZ0009-7322 http://dx.doi.org/10.1161/01.CIR.0000044384.41037.43 Google Scholar

7. T. Kume et al., “Measurement of the thickness of the fibrous cap by optical coherence tomography,” Am. Heart J. 152(4), 755.e1–755.e4 (2006). http://dx.doi.org/10.1016/j.ahj.2006.06.030 Google Scholar

8. D. Chamie et al., “Optical coherence tomography and fibrous cap characterization,” Curr. Cardiovasc. Imaging Rep. 4(4), 276–283 (2011). http://dx.doi.org/10.1007/s12410-011-9090-8 Google Scholar

9. Z. Wang et al., “Volumetric quantification of fibrous caps using intravascular optical coherence tomography,” Biomed. Opt. Express 3(6), 1413–1426 (2012).BOEICL2156-7085 http://dx.doi.org/10.1364/BOE.3.001413 Google Scholar

10. A. P. Burke et al., “Coronary risk factors and plaque morphology in men with coronary disease who died suddenly,” N. Engl. J. Med. 336(18), 1276–1282 (1997).NEJMAG0028-4793 http://dx.doi.org/10.1056/NEJM199705013361802 Google Scholar

11. R. Virmani et al., “Lessons from sudden coronary death: a comprehensive morphological classification scheme for atherosclerotic lesions,” Arterioscler. Thromb. Vasc. Biol. 20(5), 1262–1275 (2000).ATVBFA1079-5642 http://dx.doi.org/10.1161/01.ATV.20.5.1262 Google Scholar

12. R. Virmani et al., “Pathology of the vulnerable plaque,” J. Am. Coll. Cardiol. 47(8 Suppl.), C13–C18 (2006). http://dx.doi.org/10.1016/j.jacc.2005.10.065 Google Scholar

13. R. T. Lee and P. Libby, “The unstable atheroma,” Arterioscler. Thromb. Vasc. Biol. 17(10), 1859–1867 (1997).ATVBFA1079-5642 http://dx.doi.org/10.1161/01.ATV.17.10.1859 Google Scholar

14. S. J. Kang et al., “Optical coherence tomographic analysis of in-stent neoatherosclerosis after drug-eluting stent implantation,” Circulation 123(25), 2954–2963 (2011).CIRCAZ0009-7322 http://dx.doi.org/10.1161/CIRCULATIONAHA.110.988436 Google Scholar

15. T. Kubo et al., “Virtual histology intravascular ultrasound compared with optical coherence tomography for identification of thin-cap fibroatheroma,” Int. Heart J. 52(3), 175–179 (2011). http://dx.doi.org/10.1536/ihj.52.175 Google Scholar

16. O. Manfrini et al., “Sources of error and interpretation of plaque morphology by optical coherence tomography,” Am. J. Cardiol. 98(2), 156–159 (2006).AJCDAG0002-9149 http://dx.doi.org/10.1016/j.amjcard.2006.01.097 Google Scholar

17. G. van Soest et al., “Pitfalls in plaque characterization by OCT: image artifacts in native coronary arteries,” JACC: Cardiovasc. Imaging 4(7), 810–813 (2011). http://dx.doi.org/10.1016/j.jcmg.2011.01.022 Google Scholar

18. G. van Soest et al., “Atherosclerotic tissue characterization in vivo by optical coherence tomography attenuation imaging,” J. Biomed. Opt. 15(1), 011105 (2010).JBOPFO1083-3668 http://dx.doi.org/10.1117/1.3280271 Google Scholar

19. G. J. Ughi et al., “Automated tissue characterization of in vivo atherosclerotic plaques by intravascular optical coherence tomography images,” Biomed. Opt. Express 4(7), 1014–1030 (2013).BOEICL2156-7085 http://dx.doi.org/10.1364/BOE.4.001014 Google Scholar

20. C. Xu et al., “Characterization of atherosclerosis plaques by measuring both backscattering and attenuation coefficients in optical coherence tomography,” J. Biomed. Opt. 13(3), 034003 (2008).JBOPFO1083-3668 http://dx.doi.org/10.1117/1.2927464 Google Scholar

21. M. Gargesha et al., “Parameter estimation of atherosclerotic tissue optical properties from three-dimensional intravascular optical coherence tomography,” J. Med. Imaging 2(1), 016001 (2015).JMEIET0920-5497 http://dx.doi.org/10.1117/1.JMI.2.1.016001 Google Scholar

22. J. S. Choy, O. Mathieu-Costello and G. S. Kassab, “The effect of fixation and histological preparation on coronary artery dimensions,” Ann. Biomed. Eng. 33(8), 1027–1033 (2005).ABMECF0090-6964 http://dx.doi.org/10.1007/s10439-005-4854-4 Google Scholar

23. R. J. Siegel et al., “Limitations of postmortem assessment of human coronary artery size and luminal narrowing: differential effects of tissue fixation and processing on vessels with different degrees of atherosclerosis,” J. Am. Coll. Cardiol. 5(2 Pt 1), 342–346 (1985).JACCDI0735-1097 http://dx.doi.org/10.1016/S0735-1097(85)80056-5 Google Scholar

24. D. Roy et al., “3D cryo-imaging: a very high-resolution view of the whole mouse,” Anat. Rec. 292(3), 342–351 (2009). http://dx.doi.org/10.1002/ar.v292:3 Google Scholar

25. D. Roy et al., “Imaging system for creating 3D block-face cryo-images of whole mice,” Proc. SPIE 6143, 61431E (2006).PSISDG0277-786X http://dx.doi.org/10.1117/12.655617 Google Scholar

26. O. Salvado et al., “3D cryo-section/imaging of blood vessel lesions for validation of MRI data,” Proc. SPIE 6142, 614214 (2006).PSISDG0277-786X http://dx.doi.org/10.1117/12.649093 Google Scholar

27. M. S. Nguyen et al., “Ex vivo characterization of human atherosclerotic iliac plaque components using cryo-imaging,” J. Microsc. 232(3), 432–441 (2008).JMICAR0022-2720 http://dx.doi.org/10.1111/jmi.2008.232.issue-3 Google Scholar

28. E. Mehanna et al., “Volumetric characterization of human coronary calcification by frequency-domain optical coherence tomography,” Circ. J. 77(9), 2334–2340 (2013). http://dx.doi.org/10.1253/circj.CJ-12-1458 Google Scholar

29. A. Faramarzalian et al., “Ex vivo cryoimaging for plaque characterization,” JACC: Cardiovasc. Imaging 7(4), 430–432 (2014). http://dx.doi.org/10.1016/j.jcmg.2013.08.017 Google Scholar

30. O. O. Ahsen et al., “Correction of rotational distortion for catheter-based en face OCT and OCT angiography,” Opt. Lett. 39(20), 5973–5976 (2014).OPLEDP0146-9592 http://dx.doi.org/10.1364/OL.39.005973 Google Scholar

31. G. van Soest, J. G. Bosch and A. F. van der Steen, “Azimuthal registration of image sequences affected by nonuniform rotation distortion,” IEEE Trans. Inf. Technol. Biomed. 12(3), 348–355 (2008). http://dx.doi.org/10.1109/TITB.2007.908000 Google Scholar

32. T. Kawamoto and M. Shimizu, “A method for preparing 2- to 50-micron-thick fresh-frozen sections of large samples and undecalcified hard tissues,” Histochem. Cell Biol. 113(5), 331–339 (2000). http://dx.doi.org/10.1007/s004180000149 Google Scholar

33. O. Salvado and D. L. Wilson, “Removal of local and biased global maxima in intensity-based registration,” Med. Image Anal. 11(2), 183–196 (2007). http://dx.doi.org/10.1016/j.media.2006.12.001 Google Scholar


David Prabhu is a senior-level PhD candidate at Case Western Reserve University. He completed his BS degree in biomedical engineering at Case Western Reserve University. He is a Choose Ohio First Scholarship recipient and has industry experience in the field of cryo-imaging. His research interests include image processing and analysis, 3-D visualization, histology, and cryo-imaging.

Emile Mehanna is a cardiovascular medicine fellow at the University Hospitals Case Medical Center, Case Western Reserve University. His interests are intravascular cardiac imaging, notably optical coherence tomography as well as coronary artery disease plaque characterization. He has authored and coauthored more than 10 manuscripts on invasive imaging and is a reviewer for two cardiology journals.

Madhusudhana Gargesha is a senior research associate in the Department of Biomedical Engineering at Case Western Reserve University, Cleveland, Ohio, USA. He is the author of more than 15 journal papers and 20 conference publications. His current research interests include 3-D image analysis, processing, and visualization as applied to medical and biological imaging modalities including IVOCT, MRI, and block-face cryo-imaging.

Eric Brandt is currently a senior Internal Medicine Resident at the University of Chicago. He has multiple research interests within cardiology, including imaging and big data cohorts. He will be continuing his training this summer as he joins Yale-New Haven Hospital for his cardiovascular diseases fellowship starting in summer 2016.

Di Wen is a senior research associate in the Department of Biomedical Engineering at Case Western Reserve University. He received his B.S. degree in information and electronic engineering at Zhejiang University, Hangzhou, China, and Ph.D. degree in electronic engineering at Tsinghua University, Beijing, China. His research interests include computer vision, pattern recognition and image processing. He is involved in particular on face recognition, multimedia retrieval, video surveillance, and medical image analysis.

Nienke S. van Ditzhuijzen is a medical student attending a PhD program at the Erasmus University Medical Center Rotterdam. Her interests are prevention, diagnosis, and treatment of coronary artery disease, and diabetes care and intracoronary imaging, particularly coronary artery optical coherence tomography. As a PhD student, she has helped with the supervision of master students, who have all successfully completed their masters.

Daniel Chamie is an interventional cardiologist at Dante Pazzanese Institute of Cardiology, and the director of the OCT core laboratory at the Cardiovascular Research Center, Sao Paulo, Brazil. He has also served as the imaging editor of the Brazilian Journal of Invasive Cardiology since January 2014. His current major interests are the study of coronary physiology, and the application of intracoronary optical coherence tomography for the study of coronary atherosclerosis and outcomes of intracoronary devices.

Hirosada Yamamoto is the interventional cardiologist at Tokorozawa Heart Center, Japan. His interests are complex cardiovascular intervention and image analysis, specifically OCT imaging. He has certification of medical specialist of the Japanese Association of Cardiovascular Intervention and Therapeutics, Japanese Circulation Society, and Japanese Society of Internal Medicine.

Yusuke Fujino is an interventional cardiologist at New Tokyo Hospital, in Japan. His research interests include image analysis, cardiovascular imaging, and intravascular optical coherence tomography.

Ali Alian is a resident physician in diagnostic radiology at the University of Texas Southwestern Medical Center in Dallas, Texas, USA. His research interests include device development, minimally invasive surgery, and quality improvement in medical imaging.

Jaymin Patel is a PhD candidate in biomedical engineering at Johns Hopkins University. He completed his BS degree in biomedical engineering at Case Western Reserve University. His interests are image processing and registration, magnetic resonance imaging, and neuroanatomy.

Marco Costa is a professor of medicine, CWRU, Angela & James Hambrick Master Clinician in Innovation, board-certified interventional cardiologist, director of the Interventional Cardiovascular Center, and director of the Research and Innovation Center University Hospitals, Case Medical Center. He has been a leader in developing IVOCT as well as new interventional procedures. He was the first person in the US to perform clinical OCT imaging with an FDA-approved OCT imaging system from LightLab, now St. Jude.

Hiram G. Bezerra is an assistant professor of medicine, CWRU, board-certified interventional cardiologist, and medical director of the Cardiovascular Imaging Core Laboratory at University Hospitals (Core Lab), Case Medical Center. Under his Core Lab duties, he is responsible for the analysis of thousands of cardiovascular images from IVOCT as well as other modalities, e.g., intravascular ultrasound, quantitative angiography, cardiac MRI, and cardiac CT. He directs both state and federal research grants.

David L. Wilson is the Robert Herbold professor of biomedical engineering and radiology, Case Western Reserve University. His interests are image processing and analysis, cardiovascular imaging, coronary artery OCT, and cryo-imaging. He serves on NIH study sections, editorial boards, and conference committees. He has trained numerous PhD students and post docs, all of whom are quite exceptional. In addition to federal- and state-funded research activities, Professor Wilson is director of CWRU’s Interdisciplinary Biomedical Imaging NIH T32 training grant.

© The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
David Prabhu, Emile Mehanna, Madhusudhana Gargesha, Eric J. Brandt, Di Wen, Nienke S. van Ditzhuijzen, Daniel Chamie, Hirosada Yamamoto, Yusuke Fujino, Ali Alian, Jaymin Patel, Marco A. Costa, Hiram G. Bezerra, David L. Wilson, "Three-dimensional registration of intravascular optical coherence tomography and cryo-image volumes for microscopic-resolution validation," Journal of Medical Imaging 3(2), 026004 (28 June 2016). https://doi.org/10.1117/1.JMI.3.2.026004

Back to Top