1.

Introduction

According to the American Cancer Society, about 246,660 breast cancer cases were diagnosed in 2016, which is the largest population among all cancers. The estimated total death from breast cancer was almost 40,450 for 2016.¹ Early detection of breast cancer could save lives and increase treatment options. X-ray mammography is widely used for breast cancer screening; however, it misses about 10% of the cancers, especially in patients with dense breast.² Ultrasound (US) is used as an adjunct to mammography to differentiate solid from cystic lesions; however, it does not always provide needed contrast between benign and malignant solid lesions.³ MRI is frequently used for screening high-risk patients, but its overall performance is not satisfactory due to high false positive rates.²

Diffuse optical tomography (DOT) is a noninvasive technique that uses near-infrared (NIR) light to map tissue optical properties. Because water absorption in the NIR spectrum is low, the light can penetrate several centimeters inside soft tissue, for example, breast and brain. Reflected or transmitted light measured at the tissue surface is used to reconstruct tomographic images.^4,5 DOT has demonstrated huge potential in cancer diagnosis and treatment monitoring by mapping hemoglobin concentration that is related to vasculature content and tumor angiogenesis. Using multiple wavelengths, it is possible to measure oxygenated, deoxygenated, and total hemoglobin concentrations. It also provides information regarding oxygen saturation, lipid, and water concentration. These measurements could be effectively used to diagnose cancers versus benign lesions and monitor treatment response because malignant tumors typically have higher hemoglobin content as compared with benign lesions and the hemoglobin changes differ between treatment responders and nonresponders.^6–10

However, DOT suffers from intensive light scattering inside the tissue, and scattering causes uncertainty in reconstructed target location and inaccuracy of target quantification. These problems can be largely overcome using other imaging techniques to guide the DOT for localization and reconstruction. US, mammography, and MRI-guided DOT^10–12 have been investigated, and promising results have been reported. US-guided DOT has been developed by our group, and its utility in cancer diagnosis and treatment monitoring has been demonstrated from several clinical studies.^10,13,14

In the US-guided DOT approach, coregistered US images are captured and measurements of size and depth are then incorporated in DOT reconstruction as a region of interest (ROI). A dual-zone mesh image reconstruction¹⁵ is used to segment the ROI and background region with fine and coarse mesh sizes. This scheme effectively reduces the total number of voxels with unknown optical absorption for imaging reconstruction. Additionally, a total absorption of each voxel is reconstructed, and the total is then divided by the voxel size to provide absorption distributions. Because lesion absorption is higher than background in general, the total absorption, which is the product of voxel size and lesion absorption, of a smaller voxel is about the same scale of total absorption as background in a larger voxel. Therefore, the inversion is better conditioned and converges in fewer iterations compared with conventional methods that do not use a dual-mesh approach. Thus, US identified ROI is critical to guiding dual-zone mesh DOT reconstruction.

Extraction of tumor size and location from US images has been done manually, which requires experienced users to make these measurements and slows down the DOT reconstruction. Similar to other medical imaging techniques, automatic US image segmentation is a challenging task because US image contrast is low and boundaries are often not clear due to speckle. Researchers have explored several methods to obtain a reliable segmentation from medical images. These methods include operator-assisted region growing techniques,¹⁶ rule-based segmentation in which some known image primitives are used for an unsupervised segmentation,¹⁷ atlas-based image segmentation in which a known structure is searched in the image for segmentation,¹⁸ and neural network and c-mean clustering,¹⁹ which generates statistical models to classify pixels into different segments. In this article, we introduce a simple adaptive threshold-based method²⁰ that is fast on data processing and easy for implementation; moreover, it also provides comparable accuracy for DOT reconstruction as compared with manual processing. This method utilizes an image histogram to obtain an adaptive threshold for each input image. For some US images, the posterior shadow of a tumor extends to the chest wall and makes the segmentation difficult. To avoid this problem, Hough transform²¹-based line detection is used to determine the chest wall location and use it as the deep boundary of the tumor.

Twenty patients (10 benign and 10 malignant cases) are used to evaluate the performance of the segmentation method. Reconstructed absorption images are compared with a manual processing method, and similar results are obtained. To the best of our knowledge, this is the first report of an automated segmentation method using US image to guide DOT image reconstruction. The method can be modified and implemented into MRI or x-ray-guided DOT imaging reconstruction.

2.

Methods

2.2.

Extract Tumor Size and Location

To automatically detect lesion size and location for DOT reconstruction, an adaptive threshold-based segmentation method is used. For some cases, posterior shadow of the tumor is extended to the chest wall in the US images. In those cases, it is difficult to determine the tumor size because the deeper boundary of the tumor cannot be accurately determined. Under these circumstances, locations of the chest wall are determined and used as estimates of deeper boundary of the tumor. To determine the chest wall, Hough transform is used together with an edge detection method.

2.2.1.

Preprocessing

A typical coregistered US image acquired by an image capture card is given in Fig. 1(a). For reference, the vertical axis is marked as $z$-axis and the horizontal axis is marked as $x$-axis. Measurement in the $y$-axis is considered the same as the $x$-axis, assuming that lesions are symmetric in the $x$- and $y$-axes. Since the pixel intensity is the key information needed in the segmentation algorithm, the US grayscale image is automatically cropped first from the captured image before using the Hough transform and Sobel²³ edge detection method. Figure 1(b) shows the cropped US image.

Fig. 1

(a) A typical US image captured in coregistration mode and (b) cropped US image.

Depth marker detection is the next step before applying the segmentation procedure because the markers vary with depth range that depends on the user selection from the front panel of the US machine. To determine the depth markers, a binary image is generated using a fixed pixel intensity of 150 out of 256 grayscale levels as the threshold. Since the depth markers are mainly white, this pixel intensity will help to separate them from background. Then all of the white regions consisting of 3 to 50 pixels and located outside the right border of the US image are marked as depth markers. These pixel ranges are obtained by examining the available US images collected from different manufacturers. This depth marker detection procedure detects horizontal ticks along with numbers that make it suitable to use for images collected from a wide range of US machines. Figure 2 shows the captured image with automatically detected depth markers. When positions of those depth markers are known, the difference between two markers in the $z$-axis provides the number of pixels per centimeter, which are then used to convert the measured tumor size in depth into centimeter.

Fig. 2

Depth markers detected on US image.

2.2.2.

Adaptive threshold-based segmentation

To extract the required information from the US image, the first step is to segment the lesion from the rest of the image. Then the radius and center of the lesion can be measured from the segmented lesion. A single threshold point is used to separate the two zones, i.e., lesion and background. This threshold point is determined adaptively for each input image. Because US images have speckle noise, some complex segmentation techniques, such as fuzzy c-mean clustering and active contour model,²⁴ do not provide any improvement while demanding computation resources due to complex processing. Moreover, DOT does not require precise segmented information. Thus, instead of using a complex segmentation algorithm, threshold-based segmentation is used here to obtain tumor information.

Lesions in breast US images usually appear as hypoechoic masses that separate them from the background tissue. To segment a hypoechoic mass, a threshold point is set to separate the tumor from the rest of the image. US images usually have very low contrast. Histogram equalization is applied on the grayscale image. Histogram equalization stretches the input histogram over the available range, which is from 0 to 255 in grayscale, and thus increases the contrast. Then a simple procedure is followed to detect the threshold point adaptively. Since the intensity varies significantly among different images, it is best to use adaptive threshold point for every input image.

This adaptive threshold point detection procedure starts with obtaining the histogram of the US image. Figure 3(a) shows the histogram of an input image. The histogram shows a peak and a hump with a notch between them as indicated in the figure. This histogram shape is obtained from all US images after histogram equalization because of the presence of a significant amount of black (provides the peak) and gray pixels (the hump) in a US image. This notch shows the threshold for separating gray background from black tumor. To detect this point automatically, the slope of the histogram is calculated. Pixel intensity of the point when the sign of the slope has changed is considered the threshold value. In the next step, this threshold value is used to generate a binary image.

Fig. 3

(a) Histogram of a US image, threshold is marked with an arrow and (b) inserted seed on the cropped image by user.

After obtaining the threshold for the US image, a binary image is generated where tumor region is marked as black and background is white. However, tumor is not the only black zone in the binary image. To remove the unwanted black regions, the user needs to insert a seed in the approximate tumor location by clicking the tumor in the US image as shown in Fig. 3(b). If there exist multiple tumors, then multiple seeds are required to be inserted in the probable locations. Any region that does not contain a seed is discarded. Finally, only the tumor region survives. Then the MATLAB^® function “regionprop” is used to automatically measure the tumor center and radius. This information is then passed to the optical reconstruction code. The flow diagram in Fig. 4 shows the steps for the entire procedure.

Fig. 4

Flow diagram of the tumor boundary detection procedures.

2.2.3.

Chest wall detection using Hough transform

Detection of chest wall depth is not essential to obtaining tumor location and size. However, for some cases when the posterior shadow extends to the bottom of the US image, it is difficult to define the bottom of the tumor. In such cases, the chest wall location is considered the bottom of the tumor. We defined chest wall depth as the distance from the skin to the top layer of chest wall muscle. An automated chest wall depth detection method was developed and applied to the coregistered US images. Detection of the chest wall is based on the fact that chest wall muscles appear as line structures in US images [see Fig. 5(a)].²⁵ Therefore, line detection algorithms could be used for automatic detection. We chose Hough transform²¹ as a line detection method because it is simple and robust when combined with any edge detection method. Here, the Canny edge detection²⁶ method is used as an edge detection method. The binary image generated by the Canny edge detection is shown in Fig. 5(b).

Fig. 5

(a) Breast US image with chest wall marked with arrows, (b) edge detected binary image from (a), and (c) detected chest wall location on the original input image. The yellow and red stars indicate the separation points between line pieces. Green lines indicate the detected linear structures after restriction applying.

It is clear from Fig. 5(b) that, if Hough transform is applied to the edge detected image without any restriction, it will detect several unnecessary structures. For example, due to subcutaneous fat and breast tissue interfaces, some linear structures appear at the top of the US image. There are other linear structures also visible in the image. Hough transform detects all of these linear structures. To avoid these unnecessary line structures, we modeled the chest wall as a linear structure that is mainly horizontal with a small slope and it should appear at the lower half of the image. After applying Hough transform and the above-mentioned restrictions, the survived linear structures are marked in green lines as shown in Fig. 5(c). Finally, the mean value of all of the points of these detected lines is considered the chest wall depth. A flow diagram of the entire procedure is given in Fig. 6. More details on the chest wall detection method and evaluation of this method can be found in Ref. 27.

Fig. 6

Flow diagram of the chest wall detection method.

3.

Optical Reconstruction

The absorption map of each wavelength was reconstructed using the dual-mesh approach with lesion parameters obtained from coregistered US. Because the spatial resolution of diffused light is poorer than that of US, the ROI is chosen to be at least two to three times larger than that seen using US in the $x-y$ dimensions. In addition, because the depth localization of diffused light is very poor, a tighter ROI in the depth dimension is set using coregistered US. The weight matrix was computed using fitted optical properties of each patient’s normal contralateral breast. The scattered field $U_{\mathrm{sd}}$ measured from the lesion area was related to the internal total absorption coefficients $\int \mathrm{\Delta }{\mu }_{\mathrm{a}}\approx \text{voxel size}\times \mathrm{\Delta }{\mu }_{\mathrm{a}}$ using the following equation:

4.

Results

The proposed US segmentation method is evaluated in two steps. First, the US segmented reconstruction results are obtained and deviation is calculated against manually segmented reconstruction results. Second, both automated and manually segmented results are used to generate absorption maps and the corresponding hemoglobin concentration maps and are compared.

4.1.

Validation of Ultrasound Segmentation

To evaluate the performance of the US segmentation algorithm, the tumor boundary for all 20 cases was delineated by an experienced US imager. These readings are taken as standard in this study. Then the experimental results were compared with those manual measurements.

Two input images with manually marked tumor boundaries are presented in Figs. 7(a) and 8(a). In Figs. 7(b) and 8(b), segmented images using the proposed method are presented. It is clear from these figures that the segmented tumor by the proposed algorithm is comparable to the manual measurement. To obtain a quantitative evaluation of the US segmentation procedure, US images from 10 benign and 10 malignant cases are collected. Then center coordinates of the tumor and radius in both axes are measured manually. The same information is also collected from the proposed segmentation method. Then deviation is calculated between the two methods for 20 images. Comparison of the average measurements from these 20 images is given in Table 2. From the table, we found that manual measurements are slightly smaller than the proposed measurements. However, deviation from different measurements never exceeds 0.25 cm, which is the resolution of the optical reconstruction, so optical reconstruction will not be affected by this small deviation.

Fig. 7

(a) US image with manual markers to measure sizes of the tumor. The measurements were 3.1 cm in spatial direction $x$ and 1.6 cm in depth direction $z$ using manual measurements. (b) Segmented US image using the semiautomated procedure and the measurements were 3.3 cm in spatial $x$ direction and 1.6 cm in depth direction.

Fig. 8

(a) US image with manual markers to measure sizes of the tumor. The measurements were 0.88 cm in spatial direction $x$ and 0.77 cm in depth direction $z$ using manual measurements. (b) Segmented US image using the semiautomated procedure and the measurements were 0.9 cm in spatial $x$ direction and 0.73 cm in depth direction.

Table 2

Comparison between manually and semiautomatically extracted information from US images.

		Manual segmentation (cm)	Proposed segmentation (cm)	Deviation = manual — proposed (cm)
Benign cases	$z$ center position	1.49	1.56	$-0.07$l
	$x$ center position	0.14	0.19	$-0.02$
	$z$-radius	0.95	0.97	$-0.1$
	$x$-radius	1.55	1.65	$-0.05$
Malignant cases	$z$ center position	1.9	1.98	$-0.07$
	$x$ center position	0.13	0.15	$-0.03$
	$z$-radius	1.15	0.99	$-0.06$
	$x$-radius	1.34	1.4	$-0.03$

To evaluate the repeatability of the proposed US segmentation algorithm, we measured four parameters (lesion depth, $z$-radius, $x$-radius, and $x$-center) from three different sets of US images and reconstructed the corresponding total hemoglobin maps. For each case, these images were collected from the same lesion location; however, some deviation was expected because the operator intended to hold the probe still for each data set and may move a little between different data sets to obtain the best US images. For each case, mean and standard deviation are given in Table 3. This deviation for depth is $<1.5\mathrm{mm}$ and for the other three spatial measurements is smaller than 0.25 cm (image grid size) and thus does not have any major effect on optical reconstruction. As shown in the table, the maximum deviation obtained from benign cases is $4.45\mu \mathrm{M}$ and for malignant case is $2.06\mu \mathrm{M}$.

Table 3

Evaluation of the repeatability of the proposed method.

	Depth (cm)	z-radius (cm)	x-radius (cm)	x center (cm)	Total hemoglobin (μM)
Benign cases	$1.86\pm 0.042$	$1.48\pm 0.074$	$2.02\pm 0.233$	$0.47\pm 0.048$	$23.66\pm 0.905$
	$1.78\pm 0.001$	$1.57\pm 0.006$	$3.45\pm 0.189$	$0.3\pm 0.014$	$62.81\pm 0.384$
	$1.69\pm 0.013$	$1.44\pm 0.079$	$3.07\pm 0.118$	$0.45\pm 0.124$	$50.25\pm 4.452$
	$2\pm 0.006$	$0.4\pm 0.006$	$0.53\pm 0.006$	$0.58\pm 0.143$	$62.69\pm 0.362$
	$1.08\pm 0.003$	$0.39\pm 0.019$	$1.03\pm 0.070$	$-0\pm 0.026$	$42.33\pm 0.001$
	$1.18\pm 0.006$	$0.56\pm 0.009$	$1.12\pm 0.166$	$0.09\pm 0.051$	$28.26\pm 0.022$
	$1.61\pm 0.036$	$0.75\pm 0.024$	$1.61\pm 0.046$	$-0.1\pm 0.297$	$67.15\pm 0.018$
	$1.16\pm 0.009$	$0.24\pm 0.009$	$0.49\pm 0.024$	$0.15\pm 0.116$	$123.58\pm 0.317$
	$1.8\pm 0.040$	$2.52\pm 0.083$	$3.46\pm 0.114$	$-0.01\pm 0.079$	$64.11\pm 0.002$
	$1.32\pm 0.020$	$0.26\pm 0.004$	$0.48\pm 0.042$	$0\pm 0.013$	$83.98\pm 0.002$
Malignant cases	$1.35\pm 0.019$	$0.72\pm 0.012$	$0.93\pm 0.027$	$0.21\pm 0.117$	$109.78\pm 0.129$
	$2.55\pm 0.031$	$1.92\pm 0.029$	$2.84\pm 0.065$	$0.16\pm 0.077$	$172.66\pm 0.000$
	$2.97\pm 0.018$	$1.45\pm 0.036$	$2.34\pm 0.016$	$-0.63\pm 0.243$	$198.14\pm 1.734$
	$1.66\pm 0.008$	$0.88\pm 0.009$	$1.07\pm 0.031$	$0.48\pm 0.106$	$93.36\pm 2.061$
	$2.03\pm 0.029$	$0.65\pm 0.078$	$0.43\pm 0.009$	$-0.49\pm 0.035$	$95.73\pm 0.038$
	$1.45\pm 0.027$	$1.09\pm 0.009$	$1.45\pm 0.024$	$0.52\pm 0.079$	$107.23\pm 1.375$
	$2.59\pm 0.001$	$0.3\pm 0.030$	$0.55\pm 0.014$	$0.19\pm 0.028$	$135.49\pm 0.060$
	$1.94\pm 0.073$	$1.34\pm 0.137$	$2.76\pm 0.219$	$0.53\pm 0.082$	$77.77\pm 0.556$
	$1.79\pm 0.014$	$0.77\pm 0.026$	$0.81\pm 0.037$	$0.46\pm 0.099$	$156.49\pm 0.992$
	$1.57\pm 0.016$	$0.65\pm 0.000$	$0.97\pm 0.015$	$0.24\pm 0.010$	$88.84\pm 0.001$

4.2.

Validation of Optical Reconstruction

The ultimate goal of the US segmentation algorithm is to assist DOT reconstruction. In this section, performance of optical reconstruction is evaluated using tumor information extracted from both manual and proposed segmentation processes. Optical data of the same 20 patients were used to generate absorption maps for four different wavelengths. Then the absorption information is used to obtain hemoglobin concentration. Both manual and semiautomatic features are used to generate different absorption maps. In Figs. 9 and 10, absorption maps for a benign case are compared, and a malignant case is presented in Figs. 11 and 12. It is clear from these figures that the reconstructed map is almost similar. The average maximum absorption from 20 cases is compared in Table 4. From the table, we can see that the mean optical absorption obtained from manual measurement was $0.21\pm 0.06{\mathrm{cm}}^{-1}$ for malignant and $0.12\pm 0.06{\mathrm{cm}}^{-1}$ for benign cases, where for the proposed method it was $0.24\pm 0.08{\mathrm{cm}}^{-1}$ for malignant and $0.12\pm 0.055{\mathrm{cm}}^{-1}$ for benign tumors.

Fig. 9

Optical absorption maps of four wavelengths using three times of the size measured by US in $x$-dimension. Depth used in optical reconstruction is the same as US measurement. Each optical absorption map has seven image slides of 0.5 cm from the skin surface to the chest wall with 0.5 cm step in depth. Manually measured tumor information from Fig. 7(a) is used in these maps.

Fig. 10

Optical absorption maps using three times of the size identified by US in $x$-dimension. Depth used in optical reconstruction is the same as US measurement. Tumor information for these maps was extracted from Fig. 7(b).

Fig. 11

Optical absorption maps of four wavelengths using three times of US measured size in $x$ and same size as US measurement in $z$. Tumor dimension and location were extracted from Fig. 8(a) to generate these maps.

Fig. 12

Optical absorption maps using three times of US measured size in $x$ and same size as US measurement in $z$. To generate these maps, tumor information was extracted from Fig. 8(b).

Table 4

Average absorption coefficient using manual and automatically segmented tumor information.

	Malignant	Benign	Ratio
Average (standard deviation) of maximum reconstructed absorption with manual tumor segmentation (${\mathrm{cm}}^{-1}$)
740 nm	0.19 (0.08)	0.11 (0.06)	1.72
780 nm	0.22 (0.07)	0.12 (0.06)	1.83
808 nm	0.22 (0.05)	0.14 (0.08)	1.57
830 nm	0.22 (0.05)	0.13 (0.05)	1.69
Average (standard deviation) of maximum reconstructed absorption with proposed tumor segmentation (${\mathrm{cm}}^{-1}$)
740 nm	0.21 (0.08)	0.14 (0.05)	1.5
780 nm	0.25 (0.09)	0.12 (0.05)	2.08
808 nm	0.24 (0.08)	0.12 (0.06)	2
830 nm	0.24 (0.08)	0.12 (0.06)	2

Finally, Fig. 13 shows boxplots for oxygenated, deoxygenated, and total hemoglobin concentrations for the same 20 cases for both manual and proposed automated procedures. We can see from this figure that results for both techniques are almost similar. For benign cases, mean total hemoglobin concentration for all 10 cases is $58.95\pm 27.76\mu \mathrm{M}$ from manual segmentation and $58.64\pm 27.93\mu \mathrm{M}$ for the proposed automated segmentation. For malignant cases, this measurement is $115.23\pm 39.62\mu \mathrm{M}$ from manual segmentation and $114.64\pm 49.66\mu \mathrm{M}$ for the automated segmentation. Values for oxygenated hemoglobin for benign cases is $35.73\pm 20.67\mu \mathrm{M}$ for manual segmentation method and $38.32\pm 21.67\mu \mathrm{M}$ for proposed segmentation method. For malignant cases, this measurement is $72.30\pm 23.07\mu \mathrm{M}$ and $75.27\pm 27.92\mu \mathrm{M}$ for manual and proposed segmentation methods, respectively. Deoxygenated hemoglobin concentration is $35.41\pm 15.31\mu \mathrm{M}$ and $37.13\pm 16.21\mu \mathrm{M}$ for benign cases using manual and proposed segmentation methods, respectively. For malignant cases, this measurement increases to $50.26\pm 19.63\mu \mathrm{M}$ and $48.04\pm 22.88\mu \mathrm{M}$ for manual and proposed segmentation methods, respectively. Thus, the performance of the proposed feature extraction technique is quite acceptable.

Fig. 13

Comparison of hemoglobin concentration for 10 benign and 10 malignant cases.

5.

Discussion and Summary

In this work, a simple and effective US segmentation algorithm designed for assisting DOT image reconstruction is presented. This algorithm extracts tumor size and location from breast US images with minimum user interaction. It provides sufficiently accurate ROI for the DOT reconstruction; at the same time, it is very easy to implement and does not require much computational resources, thus making it ideal for real-time DOT reconstruction. Along with the threshold-based segmentation, Hough transform-based line detection is combined in the algorithm to detect chest wall location. Chest wall location is only needed when the tumor acoustic attenuation shadows the deeper boundary of the tumor.

However, this algorithm will not be able to extract information from some noisy US images when the contrast between tumor and background is very low. This work also requires limited input from users; thus, it is not fully automated. In the future, it will move toward automatic segmentation by applying a search algorithm based on the local mean²⁸ or similar seed generation method.

In conclusion, this work is one-step closer toward real-time DOT reconstruction. It eliminates the requirement for training an experienced user to provide the tumor location and size from the US images. It provides the required tumor information for the dual-mesh reconstruction with necessary accuracy. Another important feature of this proposed algorithm is that it only utilized the pixel intensities and, thus, is applicable as a segmentation approach for other imaging modalities, such as MRI-guided DOT and x-ray-guided DOT.