9 November 2017 Automated mediastinal lymph node detection from CT volumes based on intensity targeted radial structure tensor analysis
Author Affiliations +
J. of Medical Imaging, 4(4), 044502 (2017). doi:10.1117/1.JMI.4.4.044502
This paper presents a local intensity structure analysis based on an intensity targeted radial structure tensor (ITRST) and the blob-like structure enhancement filter based on it (ITRST filter) for the mediastinal lymph node detection algorithm from chest computed tomography (CT) volumes. Although the filter based on radial structure tensor analysis (RST filter) based on conventional RST analysis can be utilized to detect lymph nodes, some lymph nodes adjacent to regions with extremely high or low intensities cannot be detected. Therefore, we propose the ITRST filter, which integrates the prior knowledge on detection target intensity range into the RST filter. Our lymph node detection algorithm consists of two steps: (1) obtaining candidate regions using the ITRST filter and (2) removing false positives (FPs) using the support vector machine classifier. We evaluated lymph node detection performance of the ITRST filter on 47 contrast-enhanced chest CT volumes and compared it with the RST and Hessian filters. The detection rate of the ITRST filter was 84.2% with 9.1 FPs/volume for lymph nodes whose short axis was at least 10 mm, which outperformed the RST and Hessian filters.
Oda, Bhatia, Oda, Kitasaka, Iwano, Homma, Takabatake, Mori, Natori, Schnabel, and Mori: Automated mediastinal lymph node detection from CT volumes based on intensity targeted radial structure tensor analysis



Lung cancer is the leading cause of cancer-related deaths in the United States1 and China.2 It is also the leading cause of cancer-related deaths among men worldwide.3 There are several methods of treatment: surgery, chemotherapy, and radiotherapy. To choose the best treatment procedure, cancer staging based on the TNM staging system4 is required. Three factors are focused on in staging: T (tumor), N (lymph nodes), and distant M (metastasis). In the preoperative diagnosis of lung cancer, radiologists check mediastinal lymph nodes on computed tomography (CT) volumes. However, because lymph nodes are small and their silhouettes are not clear, they might be overlooked. To prevent medical doctors from overlooking them and to lighten their burden, a computer-aided detection system for automated lymph node detection is strongly desired.

There are various approaches for detecting lymph nodes from CT volumes: random forest statistical classifier,5 local intensity structure analyses based on Hessian matrix,67.8 or radial structure tensor (RST).9 Three-dimensional (3-D) Haar-like features are 3-D feature point detection algorithms that can detect blob-like structures in volumetric images. Barbu et al.10 introduced 3-D Haar-like features for axillary, pelvic, and abdominal lymph nodes. Feulner et al.11 utilized them for mediastinal lymph node detection. The random forest statistical classifier is a supervised machine learning technique that can be utilized for enhancing target objects in image volumes. Cherry et al.12 utilized random forest statistical classifiers for abdominal lymphadenopathy detection.

Local intensity structure analysis based on the Hessian matrix has been widely used for many algorithms of automated detection and segmentation of organs1314.15 and lesions.1617.18.19 The Hessian matrix is computed for each location and describes the local intensity structure as a blob, line, or sheet around the location, and whether it is brighter or darker than surrounding regions. The bright blob-like structure enhancement filter based on the Hessian matrix (Hessian filter) responds with a high value at the central part of the blob-like regions, which are brighter than surrounding regions. Feuerstein et al.20 proposed a mediastinal lymph node detection algorithm using this. Another algorithm proposed by Liu et al.21 is also based on Hessian analysis. Random forest5 and support vector machine (SVM)22 classifiers were introduced to improve performance. Roth et al.23 introduced deep convolutional neural networks24 for further improvement.

Another detection algorithm is through local intensity structure analysis based on RST.9 Nimura et al.25 introduced the bright blob-like structure enhancement filter based on RST (RST filter) for detecting the abdominal lymph nodes. Its benefit is that it can enhance the entire region of the target object, in contrast to the Hessian filter, which enhances only the central part of the region. The RST filter can capture the lymph node shape more properly than the Hessian filter. The features extracted can be used to determine whether each candidate region is a true positive (TP) or a false positive (FP) using machine learning techniques. However, the RST filters have so far not performed well on mediastinal lymph node detection. The current RST filter fails when tissues have largely varying intensity distributions close to the target, for instance, in the case of air and contrasting blood vessels.

Lymph nodes on CT volumes typically show the following characteristics:

  • 1. slightly higher intensity than surrounding regions,

  • 2. spherical shape,

  • 3. narrow intensity range similar to soft tissue.

To identify lymph nodes in CT images, the RST filter is designed to detect the regions with characteristics (1) and (2). However, mediastinal lymph node detection is a challenging problem in medical imaging because mediastinal lymph nodes are closely surrounded by many structures, such as contrast-enhanced blood vessels or air, as shown in Fig. 1. Although there are several works on lymph node detection on CT volumes, they fail to detect such lymph nodes. To overcome this problem, this paper proposes a new filter called the intensity targeted radial structure tensor (ITRST) filter, that is able to detect lymph nodes located around anatomical structures of extremely higher or lower intensities. The idea of the ITRST filter is to ignore extremely higher or lower intensity regions in RST computation to meet the requirement of characteristic (1). This allows us to detect lymph nodes neighboring regions with extremely high or low intensities.

Fig. 1

Intensity profile of lymph node. (a) Example of axial slice and its magnification of lymph node. Yellow represents lymph nodes. (b) Intensity profile on line segment A–B shown in (a).


The contribution of this paper is (a) to propose a new lymph node detection filter called the ITRST filter and (b) to evaluate its performance using artificial and clinical CT volumes.

In Sec. 2, we propose the automated mediastinal lymph node detection algorithm based on the ITRST filter and SVM classifier. In Sec. 3, we present two experiments showing the efficacy of the ITRST filter. The results are presented and discussed in subsequent sections.


ITRST-Based Lymph Node Detection Algorithm


ITRST Filter

This paper introduces a new blob-like structure enhancement filter called the ITRST filter, which is a modified version of the RST filter. Before explaining the ITRST filter, we will give a brief overview of the RST filter. The RST is given as the second-order tensor matrix (3×3 matrix) and is given by


where x=(x,y,z)T is the location of a voxel where the RST is computed, ri denotes an i’th search direction from x, and gi,j is a local gradient vector of I(xi,j). αi,j is an opacity, given by


αi,j={0if  |I(x)I(xi,j)|<tmin,|I(x)I(xi,j)||tmaxtmin|if  tmin|I(x)I(xi,j)|<tmax,1otherwise,
where i is the index of search directions, j is the index of search steps of each search, xij is a voxel located in the j’th search step on the i’th search direction, tmin and tmax (tmin<tmax) are parameters for controlling the sensitivity of the gradient. When an accumulated opacity βi=jαi,j1 or a search length becomes tlen or larger, a search for the i’th search direction is terminated.

Eigenvalues λ0, λ1, λ2(|λ0||λ1||λ2|) of T(x)+TT(x) represent the magnitude of the gradient directing the corresponding eigenvector around x. If all eigenvalues are negative, I(x) is brighter than the surrounding region. The larger the magnitude of the eigenvalue, the larger the gradient. The eigenvalues are utilized to enhance the bright blob-like structure regions that have the condition λ1λ2λ30 using an evaluation formula. For example, a simple evaluation formula


fblob(λ0,λ1,λ2)={|λ2||λ2||λ0|if  λ2,λ1,λ0<0,0otherwise,
was proposed by Li et al.8 Such formulas produce high responses in the bright blob-like regions.

However, if some of the radial searches incorporate regions whose intensities are extremely high or low, huge intensity gradients of some specified directions are summed into the RST T(x), according to Eq. (1). The eigenvalues calculated in such regions may become λ0λ1λ20 or λ1λ20λ0, and the responses of an evaluation formula such as Eq. (3) become low.

To prevent the effect of the huge intensity gap explained above, we propose the ITRST filter. A schematic illustration showing the difference between the RST and the ITRST filters is summarized in Fig. 2. The ITRST filter introduces the prior knowledge of the target region to prevent summing huge intensity gradients into the ITRST. Intensity gradients at the location of higher or lower intensity than the thresholds are not summed into the ITRST. The ITRST is defined by modifying Eq. (1) as


where γi,j is a function that classifies whether all points utilized for computing gi,j have the intensity within a predetermined range or not, which is defined as


γi,j={1if  tdarkI(x)tbrightfor  xN(i,j)0otherwise,,
where N(i,j) is a set of six-neighborhood voxels around xi,j. In Eq. (5), tdark and tbright are upper and lower limits of target intensity range, respectively. The parameters tdark and tbright affect the enhancement results as follows. If we lower tdark, this causes FPs of lymph nodes neighboring air regions. If we set tbright higher, this setting produces FPs of lymph nodes neighboring contrast-enhanced blood vessels (intensities of 150 to 300 H.U.). Choosing higher tdark or lower tbright makes the filter responses lower because it reduces the intensity gradients summed into the ITRST. When an accumulated opacity βi=max(jαi,j,γi,j)1 or a search length becomes tlen or larger, a search for the i’th search direction is terminated. Eigenvalues of T(x)+TT(x), λ0, λ1, λ2(|λ0||λ1||λ2|) can be utilized in the same manner as those of the RST filter.

Fig. 2

Schematic illustration showing differences between (a) RST and (b) ITRST filters. Point x is in sphere, and sphere is touching region with very low intensities. Top row represents intensity gradients that are summed into RST and ITRST, respectively. Bottom row represents magnitudes of eigenvalues λ0, λ1, λ2(|λ0||λ1||λ2|) with corresponding eigenvectors of RST and ITRST, respectively.



Mediastinal Lymph Node Detection



This section explains a mediastinal lymph node detection algorithm from CT volumes, which is based on the ITRST filter. In this algorithm, we assume that the input of the algorithm is a chest CT volume. The output is mediastinal lymph node detection results. Detection targets are the mediastinal lymph nodes with the specified least short axis or above. This is because enlarged lymph nodes have high possibility to be metastasized. The entire process consists of (1) preprocessing, (2) obtaining candidate regions by thresholding against the responses of the ITRST filter, and (3) FP reduction using machine learning.

The input volume I is a chest CT volume. The ground-truth binary volume Ig of mediastinal lymph nodes is required if I is utilized for training only. The output is a binary volume Iout, which has lymph node regions denoted by the value 1. We describe the size of each lymph node using principal components analysis. We focus on the detection of lymph nodes above a specified size, defined by a short axis length of at least rtarget (mm).



First, we interpolate an input volume to generate an isotropic resolution volume. We apply the cubic interpolation on I to obtain a volume with isotropic resolution wreso  (mm)×wreso  (mm)×wreso  (mm) per voxel.26 Furthermore, we apply a Gaussian smoothing filter with standard deviation σsmooth (mm) for reducing noise and making local gradients more stable. We denote the preprocessed input volume as I.


Target region of detection

We restrict the target region for lymph node detection to inside the mediastinal region, which we define as the area between the lungs. A lung region Alung is segmented by an automatic segmentation algorithm. We use a lung segmentation algorithm similar to Hu et al.27

First, we obtain air regions Aair in I as regions having lower intensities than a given threshold tair and not touching the boundary of the input CT volume. Then, we select the first and second largest connected components from the air regions detected. If the second largest component of the air regions is less than 20% of the largest one, we choose the largest component of the air regions as Alung. Finally, the mediastinum region Amedia is obtained by Alung using Procedure 1. The function max(a) represents the maximum value of a in this procedure.

Procedure 1

Segmentation of mediastinum region Amedia

Input: lung region Alung
forz=0 to max(z)do
fory=0 to max(y)do
  forx=0 to max(x)1do
   end if
  end for
   forx=0 to max(x)1do
    end if
    end if
   end for
  end if
end for
end for
Output: mediastinum region Amedia

We assume that the x-axis of the input volume I corresponds with the right-to-left direction of the body, the y-axis corresponds with the front-to-back direction, and z-axis corresponds with the head-to-foot direction. We scan the lung region Alung from each pair of (y,z) toward the right-to-left direction (along the x-axis). For each scan, we check whether no fewer than two components exist in the scan line. This means that there are right and left lung regions in the scan line. If no fewer than two components exist, we fill the gap between each component with the value 1.


Initial lymph node detection using ITRST filter

We obtain the candidate regions using the ITRST filter. First, we apply the ITRST filter with the evaluation Eq. (3) to I. Since each point of I has one response, we obtain the volume of filter response F. Because responses of the ITRST filter sometimes become zero in the blob-like regions such as an individual hole, we apply a median filter of whole×whole×whole (voxels) to F to normalize such points. Then, we perform thresholding on the result of the median filter F as


with the threshold value tblob. Regions of connected components whose volumes are less than that of the sphere with radius tsmall (mm) and ones that are not touching the mediastinum region Amedia are eliminated. The remaining are denoted as candidate regions.


FP reduction

The ITRST filter detects many FP regions as initial lymph node candidate regions. The SVM classifier is utilized to classify each candidate region into TP or FP regions. (10+7|D|)-dimensional features are utilized for each candidate region, as shown in Table 1. D is the permutation of radii utilized for computation of features related to the intensity. The number of elements of D is |D|, index of D is p(1p|D|), and one of its elements is written as dD. When d>0  mm, the target region for computing the intensity features is obtained by dilation of the candidate region, by using a structure element of a sphere whose radius is d  mm. The target region represents a neighbor of the candidate region, and larger values of d make the target region thicker. When d=0  mm, the target region is the same as the candidate region. Features are computed for both the training step and the testing step, and utilized as follows.

  • Training step

    Feature vectors obtained from candidate regions of all volumes in the training dataset are used for training of the SVM.22 The set of lymph node regions whose short axis is at least rtargetwtrain  mm in the ground truth Ig is written as G. rtarget is a parameter representing the minimum length of the short axis of target lymph nodes. wtrain(0<wtrain<1) is the parameter for setting the smallest size of a lymph node that is utilized for training. If the center of a candidate region is in one of the lymph node regions of G, its feature vector is utilized as a positive sample. If the center of a candidate region is outside of the lymph node regions, its feature vector is utilized as a negative sample.

  • Testing step

    A candidate region extracted from a test volume is classified into TP or FP with its feature vector by using an SVM trained as above. If a candidate region is predicted as TP, the values of the output binary volume Iout are set to 1 in the candidate region.

Table 1

Features for FP reduction step.

u1Volume (mm3)
u2Surface area (mm2)
u4Maximum length from contour (mm)
u5Length of long axis (mm)
u6Length of short axis (mm)
Locationu8Normalized y-coordinate of Ck in bounding box of lung
Intensityu13+7(p1)Maximum of I in the target region. See Sec. 2.2.5 for details.


Materials and Methods


ITRST Filter



An artificially generated volumetric image is used for evaluation of the ITRST filter. This volume includes seven objects imitating lymph nodes, three objects imitating contrast-enhanced blood vessels, and three objects imitating air regions. Figures 3(a) and 3(b) show the blueprint and one slice of the artificially generated volume containing the synthetic objects, respectively. This volume contains one isolated sphere, three spheres overlapping with 300 H.U. square poles, and three spheres overlapping with 1000  H.U. square poles. The background of the volume is 0 H.U. First, the spheres with a diameter of 15 mm are drawn as uniform of 50 H.U., and Gaussian smoothing of σ=1.0  mm is applied to make the spheres similar to lymph nodes of real CT volumes. After that square poles with thickness of 15 mm and length of 50 mm are drawn.

Fig. 3

Synthetic examples of solid objects. (a) Blueprint of artificially generated volume. Slice contains centers of all spheres. (b) Slice containing one isolated sphere and six spheres touching bright (300 H.U.) or dark (1000  H.U.) square poles.



Filter response evaluation

We apply the ITRST, RST, and Hessian filters to the artificially generated volume explained above. First, we examine the filter responses. In this experiment, we obtain the eigenvalue profile on the two spheres shown in Fig. 3(b). The line A–B in Fig. 3(b) crosses one of the spheres overlapping a dark region. The line C–D is one of the spheres overlapping a bright region.

Parameters used in the experiments are shown in Table 4(a). We set tbright=100  H.U. and tdark=100  H.U. since the spheres have comparability with lymph nodes, which range from 100 to 100 H.U. We also set tlen=15  mm as the upper limit of the radius of lymph nodes to be detected in real CT volumes.


Mediastinal Lymph Node Detection



Forty-seven chest CT volumes were prepared for the experiments on mediastinal lymph node detection. These volumes were authorized by the ethics committee of Nagoya University Hospital. The specifications of the volumes are shown in Table 2. We evaluate the performance of lymph nodes for a range of minimum sizes: the short axis is at least rtarget{10,7.5,5  mm}. Ground-truth data are a set of mediastinal lymph node labels. Two technical researchers who have sufficient knowledge of lymph nodes first manually traced lymph node candidate regions on the CT volumes. Then, an expert radiologist confirmed these traced data including missing lymph nodes on CT slices. Table 3 shows the number of lymph nodes of each size category.

Table 2

Specification of CT volumes used in experiments on mediastinal lymph node detection.

Number of volumes47
DeviceAquilion 64, Toshiba
Reconstruction functionFC11
Size512×512×(338463) voxels
Resolution(0.6250.782)×(0.6250.782)×(0.7990.801)  mm3

Table 3

Number of lymph nodes categorized by short axes.

Size categoryNumber of lymph nodes
[10 mm, )57
[7.5 mm, 10 mm)62
[5 mm, 7.5 mm)145
[3 mm, 5 mm)284


Initial detection performance evaluation

We compute FROC curves of initial detection results by changing the parameter tblob, for comparing the ITRST, RST, and Hessian filters. The filter output is binarized at different thresholds (tblob=20, 40, 80, and 160 for the ITRST filter, tblob=20, 40, 80, and 160 for the RST filter, and tblob=2000, 4000, 8000, and 16,000 for the Hessian filter), as explained in Sec. 2.2.4.

Each point on FROC curves represents the averages of the detection rate and the number of FPs/volume among all volumes. The corresponding error bars represent the standard deviation of the detection rate. Our detection targets are mediastinal lymph nodes whose short axes are at least rtarget (mm). Each mediastinal lymph node is classified and counted as TP or FN. If at least one region produced by the filter overlaps with a mediastinal lymph node of the detection target, the lymph node is counted as TP. The detection rate of each volume is defined as

(Detection rate of each volume)=(Number ofTPs)(Numberofdetectiontargets).

The number of FPs in each volume is the count of regions produced by the filter that do not overlap with any lymph nodes or lung cancers.

The FROC curves are drawn for each value (5, 7.5, and 10 mm) of the least short axis parameter rtarget. Parameters of tbright, tdark, and tlen are the same as the experiment of Sec. 3.1. Other parameters are set empirically [Table 4(b)]. The air region segmentation threshold tair is set as 200  H.U. This threshold is set to be sufficiently lower than lymph nodes or surrounding soft tissues, which have around 100  H.U. or above. Isotropic resolution parameter wreso is set as 0.625 mm. This equals the smallest pixel size of axial slices of the CT volumes (Table 2). Standard deviation of Gaussian smoothing filter σsmooth is set as 1 mm. This setting is good for reducing noise on chest CT volumes without severely blurring edges. The parameter whole, the size of median filter applied to the output of the ITRST filter, is set as three voxels. This is the smallest size of median filter.

Table 4

Constant parameter values. (a) ITRST filter. (b) Mediastinal lymph node detection: initial detection evaluation. (c) Mediastinal lymph node detection: overall detection performance evaluation. Note that rtarget, tblob, and wF are not shown here since they are varied for evaluating the performance in different conditions.

(a) ITRST filter
tbright100 H.U.Upper limit of intensity target range of ITRST filter
tdark100  H.U.Lower limit of intensity target range of ITRST filter
tlen15 mmMaximum search length of ITRST filter
(b) Mediastinal lymph node detection: Initial detection evaluation
tbright100 H.U.Upper limit of intensity target range of ITRST filter
tdark100  H.U.Lower limit of intensity target range of ITRST filter
tlen15 mmMaximum search length of ITRST filter
tair200  H.U.Threshold for dividing air and other regions
wreso0.625 mmIsotropic resolution at which input volume is interpolated
σsmooth1 mmStandard deviation of Gaussian smoothing filter
whole3 voxelsSize of median filter applied for output of ITRST filter
(c) Mediastinal lymph node detection: Overall detection performance evaluation
wtrain0.5Tolerance for using feature vectors extracted from candidate regions of smaller lymph nodes than rtarget for training SVM
D{0,1,2}  mmWidth of regions for computing feature values regarding intensity


Overall detection performance evaluation

We compute the FROC curves as the overall performance. This performance includes FP reduction by SVM. An FROC curve is obtained by changing the weighting parameter wF of negative samples for the SVM classifier22 utilized in the FP reduction step (explained in Sec. 2.2.5). Leave-one-out cross validation is conducted to evaluate the performance of FP reduction for each volume. The SVM classifier is tested with the data not used in the training process.

We also conduct a statistical test (Fisher’s exact test) of the detection rate obtained by the ITRST filter and the others. For fair comparison of detection rate between the filters, we draw the FROC curve with various values of wF and estimate the detection rate at the point of 10.0 FPs/volume on the FROC curve.

The threshold for the filter output is chosen as tblob=20 for the ITRST and RST filters, and tblob=2000 for the Hessian filter, since these settings of tblob gave the highest initial detection rate with each filter. To compute the FROC curves, the weighting parameter wF is changed as 0.025, 0.05, 0.075, 0.10, 0.125, 0.15, 0.20, 0.25, 0.30, 0.40, and 0.50. Other parameters utilized for the FP reduction step are set empirically [Table 4(c)]. The parameter wtrain for tolerance of using feature vectors extracted from candidate regions of lymph nodes smaller than the target for training SVM is set as 0.5 for preventing false negatives of lymph nodes whose short axis is almost the same as rtarget. Permutation D representing the width of regions for computing feature values regarding intensity is set as {0,1,2}mm for focusing on the inside and neighboring regions of each candidate region. The LIBSVM 3.1728 library is utilized as an SVM implementation.




ITRST Filter

The responses of the ITRST, RST, and Hessian filters for the artificially generated volume are shown in Figs. 4(a)4(c), respectively. The responses were higher in most of the sphere regions than those of the RST and Hessian filters, despite the overlapping of square poles.

Fig. 4

Responses for artificially generated volume. (a) ITRST filter. (b) RST filter. (c) Hessian filter. Color scheme is same as (a). Blue represents low response (around 1), yellow represents medium response (around 125), and red represents high response (around 250).


The eigenvalue profiles obtained by the ITRST, RST, and Hessian filters are shown in Figs. 5(a)5(c), respectively. By using the ITRST filter, all eigenvalues were negative, and their magnitudes do not differ much in the entire part of both spheres. By using the RST filter, λ1 became far smaller or larger than other eigenvalues in the spheres touching the bright and dark square poles, respectively. Eigenvalues of the Hessian filter also became positive in the parts near the square poles.

Fig. 5

Eigenvalue profiles on lines A–B and C–D shown in Fig. 3(b). (a) ITRST filter. (b) RST filter. (c) Hessian filter.



Mediastinal Lymph Node Detection


Initial detection evaluation

The FROC curves of initial detection are shown in Fig. 6. As shown in Table 5, a higher detection rate was achieved by the proposed algorithm (ITRST filter) than by the RST filter. For instance, when rtarget=10  mm and tblob=20, 97.1% of lymph nodes were detected with 692.1 FPs/volume by the proposed algorithm (ITRST filter). Using the RST (tblob=20) or Hessian filters (tblob=2000), 75.4% or 91.1% were detected with 377.8 or 683.2 FPs/volume, respectively. Examples of the detection results are shown in Fig. 8.

Fig. 6

FROC curves obtained after initial detection (tblob for 20.0, 40.0, 80.0, and 160.0 for ITRST and RST filters, and tblob for 2000, 4000, 8000, and 16,000 for Hessian filter). (a) rtarget=5  mm. (b) rtarget=7.5  mm. (c) rtarget=10  mm.


Table 5

Comparison of lymph node detection performances. Note that symbol * represents performance shown in publication. Their experiments were performed using different datasets or criteria from ours.

AlgorithmLeast short axis (mm)Initial detectionAfter FP reduction
Detection rate (%)FPs/volumeDetection rate (%)FPs/volume
Proposed algorithm594.0±13.4679.6±83.668.4±25.710.0±5.2
RST filter556.0±26.7373.5±80.744.0±24.66.6±4.2
Hessian filter575.8±21.4675.8±117.054.9±25.611.9±5.3
Proposed algorithm7.597.4±9.9686.8±84.372.8±29.610.1±5.3
RST filter7.568.7±32.0373.7±80.455.8±33.46.6±4.3
Hessian filter7.585.5±21.1675.6±118.168.6±33.511.7±5.3
Proposed algorithm1097.1±17.1692.1±82.684.2±31.09.1±5.1
RST filter1075.4±37.6377.8±77.263.9±41.46.3±4.2
Hessian filter1091.1±22.2683.2±121.778.2±35.311.1±5.1
*Roth et al.2310846
*Liu et al.2910808
*Feulner et al.111060.96.1


Overall detection performance evaluation

Table 5 and Fig. 7 show the overall performance calculated from the output of the mediastinal lymph node detection algorithm, with parameters tblob=20 (tblob=2000 for Hessian filters) and wF=0.075. For example, when rtarget=10  mm, 84.2% of lymph nodes were detected with 9.1 FPs/volume by the proposed algorithm (ITRST filter). Table 6 displays the results of Fisher’s exact test at 10.0 FPs/volume. Performance of the proposed algorithm was not always significantly better. It was shown that detection rates of ITRST and RST filters were significantly different (p<0.05) with all settings (5, 7.5, and 10 mm) of the least short axis. On the other hand, detection rates of the ITRST and Hessian filters were significantly different when the least short axis was 5 mm.

Fig. 7

FROC curves obtained after FP reduction (wF for 0.025, 0.05, 0.075, 0.10, 0.125, 0.15, 0.20, 0.25, 0.30, 0.40, and 0.50) with tblob=20 for ITRST and RST filters, and tblob=2000 for Hessian filter. (a) rtarget=5  mm. (b) rtarget=7.5  mm. (c) rtarget=10  mm.


Table 6

Fisher’s exact test among detection rate at 10.0 FPs/volume.

Least short axis (mm)Detection rate (%)p-value




ITRST Filter

It is clear that responses of the ITRST filter were higher in most of the sphere regions than those of the RST filter, even if the square poles were overlapping, as shown in Fig. 4. It suppressed the negative effect of the regions with much higher or lower intensity than the detection target. The Hessian filter yielded positive values only on some parts of the spheres.

The magnitude of λ1 computed from the ITRST filter was similar to λ2 and λ3, and it was negative on almost all of A–B or C–D, as shown in Fig. 5(a). Huge intensity gradients were not summed into the ITRST, and the eigenvalues followed the condition of the bright blob-like structure. In contrast, as shown in Fig. 5(b), the magnitude of λ1 computed from the RST filter was far larger than λ2 and λ3 in the sphere. This is because huge intensity gradients directed from the sphere to the square pole were accumulated into the RST. Figure 5(c) shows the magnitude of λ1 computed from the Hessian filter, which becomes large in the part near the square poles in the Hessian filter. The eigenvalues did not follow the condition of the bright blob-like structure (λ1λ2λ30) in the part with large magnitude of λ1, and the responses became low according to Eq. (3).

By comparing the FROC curves of initial detection shown in Fig. 6, it is clear that the ITRST filter had a higher detection rate for both large and small lymph nodes than the RST filter. Results after FP reduction of the proposed algorithm (ITRST filter) were also better than those of the RST filter. The ITRST filter is more useful than the RST filter for mediastinal lymph node detection.


Mediastinal Lymph Node Detection


Efficacy of ITRST filter

The lymph node shown in Fig. 8(a) was properly detected by the proposed algorithm (ITRST filter) and the Hessian filter, while the RST filter was not able to detect it. This was likely due to the presence of contrast-enhanced blood vessels and the air region adjacent to the lymph node. In contrast to the RST filter, the ITRST filter reduced the impact of the large intensity gradients around the lymph node so that the eigenvalues still followed the condition of the bright blob-like structure. This lymph node was not surrounded entirely by extremely high or low intensity regions, so some intensity gradients derived from soft tissue could still be utilized to descibe the bright blob-like structure.

Fig. 8

Examples of detection results (a)–(e) with parameters tblob=20 and rtarget=10  mm. Yellow denotes ground truth. Cyan denotes TP detection. Red denotes FP detection. Green represents detection of small or hilar lymph nodes. First two rows represent candidate regions and after FP reduction using ITRST filter. Third and fourth rows represent results of RST filter. Fifth and sixth rows are of Hessian filter.


The lymph node shown in Fig. 8(b) was detected initially by all algorithms. The candidate region obtained by the RST filter was poorly segmented because of the negative effect of the contrast-enhanced blood vessels and the air, and it was removed by SVM. However, the candidate region obtained by the ITRST filter covered most of the lymph node region, and it was classified as a lymph node. The ITRST filter prevented the negative effect of surrounding regions and contributed to accurate classification. Although the candidate region obtained by the Hessian filter was a little smaller than that of the ITRST filter, it was also properly classified.


False negatives still produced

Some lymph nodes were still missed by the ITRST filter, as shown in the FROC curves in Fig. 6. This is because they tend to be sandwiched by extremely high or low intensity regions. For instance, the lymph node shown in Fig. 8(c) could not be detected by the ITRST or the RST filters. It was sandwiched by the contrast-enhanced blood vessels and the air region, and a very small amount of soft tissue was touching the lymph node. Most gradient vectors around it were not summed into the ITRST, so the magnitude of eigenvectors did not follow the condition of the bright blob-like structure. In contrast to the ITRST or RST filters, the Hessian filter did not strongly suffer from intensity differences between lymph nodes and neighboring regions. The small candidate region detected by the Hessian filter was finally classified into lymph node class after the FP reduction process. It remains as future work to improve the ITRST filter such that it can segment lymph nodes surrounded by extremely high or low intensity regions.

The lymph node shown in Fig. 8(d) was initially detected by the ITRST filter, but it was removed by the SVM classifier. To prevent generating such false negatives, we will improve the classification accuracy by introducing deep learning techniques in future work. Note that the candidate region of the ITRST filter was properly classified as a lymph node with wF=0.025.


Promise for application to segmentation

As shown in Fig. 8(e), some lymph nodes are detected by all filters (ITRST, RST, and Hessian). However, the ITRST filter produced more proper segmentation results of lymph nodes than other filters. In the future, the ITRST filter can be improved for application to segmentation of lymph nodes, not only for detection. This will assist radiologists in measuring the size and shape of each lymph node.


Lung Area Segmentation

We have extracted mediastinum regions from CT volumes by extracting lung regions. There is some possibility to fail in lung region segmentation in a pathological lung with cancer, as shown in Fig. 9. However, that does not affect the subsequent processes since lung segmentation is only for obtaining the mediastinum region sandwiched between the lungs. The lung cancer region is merged into the target region.

Fig. 9

Target region extraction on volume with lung cancer region. (a) Input volume. (b) Lung cancer region. (c) Lung region. (d) Target region.




In this paper, we proposed a mediastinal lymph node algorithm based on the ITRST filter. The conventional RST filter cannot detect some lymph nodes because of neighboring regions of lymph nodes, which have extremely high or low intensities. We proposed the ITRST filter by modifying the RST filter to prevent such negative effects by introducing knowledge about the intensity range of the detection target. It allows us to exclude neighboring regions for computing the filter response and increase the detection rate of lymph nodes.

We evaluated the efficacy of the ITRST filter by applying it to both an artificially generated volume and chest CT volumes. In an experiment on synthetic data, the ITRST filter produced high responses in the spheres neighboring bright or dark square poles, while responses of the RST filter were very low or zero. These results showed that the ITRST filter can prevent the negative effect caused by such neighboring regions, in contrast to the RST filter.

Furthermore, experimentation with real clinical images for mediastinal lymph node detection showed that the ITRST filter outperformed the RST filter. This is because most of the mediastinal lymph nodes adjacent to air or contrast-enhanced blood vessels in the chest CT volumes can be detected using the ITRST filter. The detection performance after FP reduction is also better than the RST filter. The proposed ITRST filter could potentially be used for detection of other organs or tissues of interest in medical imaging.


There is no conflict of interest in this paper.


This work was supported by JSPS KAKENHI Grant Numbers 26108006, 26560255, 25242047, and 17H00867. K. Mori and J. Schnabel acknowledge funding from the Royal Society International Exchanges award IE141556, and JSPS Bilateral Joint Research Project “Oncological Diagnostic and Interventional Assistance System Based on Multi-modality Medical Image Processing.” J. Schnabel and K. Bhatia acknowledge funding from the TOHETI award by the Guy’s and St. Thomas’ Charity.


1. R. L. Siegel, K. D. Miller and A. Jemal, “Cancer statistics, 2016,” CA-Cancer J. Clin. 66(1), 7–30 (2016). http://dx.doi.org/10.3322/caac.21332 Google Scholar

2. W. Chen et al., “Cancer statistics in China, 2015,” CA-Cancer J. Clin. 66(2), 115–132 (2016). http://dx.doi.org/10.3322/caac.21338 Google Scholar

3. L. A. Torre, R. L. Siegel and A. Jemal, “Lung cancer statistics,” Adv. Exp. Med. Biol. 893, 1–19 (2016). http://dx.doi.org/10.1007/978-3-319-24223-1_1 Google Scholar

4. S. Mirsadraee et al., “The 7th lung cancer TNM classification and staging system: review of the changes and implications,” World J. Radiol. 4(4), 128–134 (2012). http://dx.doi.org/10.4329/wjr.v4.i4.128 Google Scholar

5. L. Breiman, “Random forests,” Mach. Learn. 45(1), 5–32 (2001).MALEEZ0885-6125 http://dx.doi.org/10.1023/A:1010933404324 Google Scholar

6. A. F. Frangi et al., “Model-based quantitation of 3-D magnetic resonance angiographic images,” IEEE Trans. Med. Imaging 18(10), 946–956 (1999).ITMID40278-0062 http://dx.doi.org/10.1109/42.811279 Google Scholar

7. Y. Sato et al., “Tissue classification based on 3D local intensity structures for volume rendering,” IEEE Trans. Visual Comput. Graphics 6(2), 160–180 (2000). http://dx.doi.org/10.1109/2945.856997 Google Scholar

8. Q. Li, S. Sone and K. Doi, “Selective enhancement filters for nodules, vessels, and airway walls in two-and three-dimensional CT scans,” Med. Phys. 30(8), 2040–2051 (2003).MPHYA60094-2405 http://dx.doi.org/10.1118/1.1581411 Google Scholar

9. R. Wiemker et al., “A radial structure tensor and its use for shape-encoding medical visualization of tubular and nodular structures,” IEEE Trans. Visual Comput. Graphics 19(3), 353–366 (2013). http://dx.doi.org/10.1109/TVCG.2012.136 Google Scholar

10. A. Barbu et al., “Automatic detection and segmentation of lymph nodes from CT data,” IEEE Trans. Med. Imaging 31(2), 240–250 (2012).ITMID40278-0062 http://dx.doi.org/10.1109/TMI.2011.2168234 Google Scholar

11. J. Feulner et al., “Lymph node detection and segmentation in chest CT data using discriminative learning and a spatial prior,” Med. Image Anal. 17(2), 254–270 (2013). http://dx.doi.org/10.1016/j.media.2012.11.001 Google Scholar

12. K. M. Cherry et al., “Abdominal lymphadenopathy detection using random forest,” Proc. SPIE 9035, 90351G (2014).PSISDG0277-786X http://dx.doi.org/10.1117/12.2043837 Google Scholar

13. H. Shikata, E. A. Hoffman and M. Sonka, “Automated segmentation of pulmonary vascular tree from 3D CT images,” Proc. SPIE 5369, 107–116 (2004).PSISDG0277-786X http://dx.doi.org/10.1117/12.537032 Google Scholar

14. S. Kawajiri et al., “Automated segmentation of hepatic vessels in non-contrast x-ray CT images,” Radiol. Phys. Technol. 1(2), 214–222 (2008). http://dx.doi.org/10.1007/s12194-008-0031-4 Google Scholar

15. A. H. Foruzan et al., “A Hessian-based filter for vascular segmentation of noisy hepatic CT scans,” Int. J. Comput. Assisted Radiol. Surg. 7(2), 199–205 (2012). http://dx.doi.org/10.1007/s11548-011-0640-y Google Scholar

16. Y. Masutani, H. MacMahon and K. Doi, “Computerized detection of pulmonary embolism in spiral CT angiography based on volumetric image analysis,” IEEE Trans. Med. Imaging 21(12), 1517–1523 (2002).ITMID40278-0062 http://dx.doi.org/10.1109/TMI.2002.806586 Google Scholar

17. I. Isgum, B. van Ginneken and M. Olree, “Automatic detection of calcifications in the aorta from CT scans of the abdomen,” Acad. Radiol. 11(3), 247–257 (2004). http://dx.doi.org/10.1016/S1076-6332(03)00673-1 Google Scholar

18. S. H. Kim et al., “Computer-aided detection of colonic polyps at CT colonography using a Hessian matrix-based algorithm: preliminary study,” Am. J. Roentgenol. 189(1), 41–51 (2007).AJROAM0092-5381 http://dx.doi.org/10.2214/AJR.07.2072 Google Scholar

19. Z. Shi et al., “A computer aided pulmonary nodule detection system using multiple massive training SVMs,” Appl. Math. Inf. Sci. 7(3), 1165–1172 (2013). http://dx.doi.org/10.12785/amis/070339 Google Scholar

20. M. Feuerstein et al., “Mediastinal atlas creation from 3-D chest computed tomography images: application to automated detection and station mapping of lymph nodes,” Med. Image Anal. 16(1), 63–74 (2012). http://dx.doi.org/10.1016/j.media.2011.05.005 Google Scholar

21. J. Liu et al., “Mediastinal lymph node detection and station mapping on chest CT using spatial priors and random forest,” Med. Phys. 43(7), 4362–4374 (2016).MPHYA60094-2405 http://dx.doi.org/10.1118/1.4954009 Google Scholar

22. C. Cortes and V. Vapnik, “Support-vector networks,” Mach. Learn. 20(3), 273–297 (1995).MALEEZ0885-6125 http://dx.doi.org/10.1007/BF00994018 Google Scholar

23. H. R. Roth et al., “A new 2.5D representation for lymph node detection using random sets of deep convolutional neural network observations,” Lect. Notes Comput. Sci. 8673, 520–527 (2014). http://dx.doi.org/10.1007/978-3-319-10404-1_65 Google Scholar

24. A. Krizhevsky, I. Sutskever and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Int. Conf. on Advances in Neural Information Processing Systems, pp. 1097–1105 (2012). Google Scholar

25. Y. Nimura et al., “Automated abdominal lymph node segmentation based on RST analysis and SVM,” Proc. SPIE 9035, 90352U (2014).PSISDG0277-786X http://dx.doi.org/10.1117/12.2043349 Google Scholar

26. R. Keys, “Cubic convolution interpolation for digital image processing,” IEEE Trans. Acoust. Speech Signal Process. 29(6), 1153–1160 (1981).IETABA0096-3518 http://dx.doi.org/10.1109/TASSP.1981.1163711 Google Scholar

27. S. Hu, E. A. Hoffman and J. M. Reinhardt, “Automatic lung segmentation for accurate quantitation of volumetric x-ray CT images,” IEEE Trans. Med. Imaging 20(6), 490–498 (2001).ITMID40278-0062 http://dx.doi.org/10.1109/42.929615 Google Scholar

28. C.-C. Chang and C.-J. Lin, “LIBSVM: a library for support vector machines,” ACM Trans. Intell. Syst. Technol. 2(3), 27:1–27:27 (2011). http://dx.doi.org/10.1145/1961189 Google Scholar

29. J. Liu et al., “Mediastinal lymph node detection on thoracic CT scans using spatial prior from multi-atlas label fusion,” Proc. SPIE 9035, 90350M (2014).PSISDG0277-786X http://dx.doi.org/10.1117/12.2043737 Google Scholar


Hirohisa Oda received MEng from Nagoya University in 2014. After working in industry, he started a PhD program of Nagoya University from 2015. His specializations are image processing and machine learning. His research interests are computer-aided diagnosis and microfocus x-ray CT for cardiac image processing.

Kanwal K. Bhatia received her PhD in computer science from Imperial College London in 2007. After a period in industry, she returned to Imperial College London from 2010 to 2015 to work as a research associate in intelligent cardiovascular imaging, and from 2015 to 2017 she worked as a research fellow at the Department of Biomedical Engineering, King’s College London, on deep learning methods for lung cancer detection. In 2017, she returned to work in industry.

Masahiro Oda is an assistant professor at the Graduate School of Informatics, Nagoya University. On April 2004, he enrolled at the Department of Media Science, Graduate School of Information Science, Nagoya University. On March 2009, he graduated from PhD course of the Department of Media Science, Graduate School of Information Science, Nagoya University. His research interests are medical image processing, computer-aided diagnosis, and computer-assisted interventions.

Takayuki Kitasaka received his BS degree in information engineering and his PhD in engineering from Nagoya University, Japan, in 1997 and 2002. He was an assistant professor at the School of Engineering, Nagoya University from 2003 to 2008. He was a lecturer of Aichi Institute of Technology from 2008 to 2011. Since 2011, he has been an associate professor of Aichi Institute of Technology.

Shingo Iwano is an associate professor at the Department of Radiology, Nagoya University Graduate School of Medicine. He received his MD and PhD degrees from Nagoya University in 1993 and 2001. His research interests are chest radiology and computer-aided diagnosis.

Masaki Mori is a director of Department of Respiratory Medicine, Sapporo Kosei-General Hospital. He received his MD and PhD degrees in medicine from the Sapporo Medical University in 1979 and 1989, respectively. His research interests are medical image processing and computer-aided diagnosis.

Hiroshi Natori is a professor emeritus of Sapporo Medical University, School of Medicine since 2005. He received his MD and PhD degrees in medicine from Sapporo Medical University. His major is in respiratory medicine. His research interest is the analysis of three dimensional architectures and function of the lung. He served as honorary director of Nishioka Hospital of the Keiwakai Social Medical Corporation, Sapporo.

Julia A. Schnabel is a professor in the Department of Biomedical Engineering, King’s College London since 2015. She received her PhD from University College London in 1998. She is on the editorial board of Medical Image Analysis, and an associate editor of IEEE Transactions of Medical Imaging, and IEEE Transactions on Biomedical Engineering. Her research interests include complex motion modelling and machine/deep learning in oncological, cardiovascular, perinatal and neurological imaging, across multiple imaging modalities.

Kensaku Mori is a professor of Graduate School of Information Science, Nagoya University and the director of Information Technology Center of Nagoya University and MICCAI fellow. He received MEng in information engineering and PhD in information engineering from Nagoya University in 1994 and 1996, respectively. He was also involved in many international conference organizations, including SPIE Medical Imaging, CARS, and MICCAI, as a general chair or program committee members.

Biographies of the other authors are not available.

© The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Hirohisa Oda, Kanwal K. Bhatia, Masahiro Oda, Takayuki Kitasaka, Shingo Iwano, Hirotoshi Homma, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori, Julia A. Schnabel, Kensaku Mori, "Automated mediastinal lymph node detection from CT volumes based on intensity targeted radial structure tensor analysis," Journal of Medical Imaging 4(4), 044502 (9 November 2017). https://doi.org/10.1117/1.JMI.4.4.044502 Submission: Received 4 April 2017; Accepted 16 October 2017
Submission: Received 4 April 2017; Accepted 16 October 2017

Lymphatic system

Target detection

Detection and tracking algorithms

Computed tomography

Optical spheres


Digital filtering

Back to Top