There are length of Earth levees in the United States, and even more throughout the world with various designs. According to a Governing.com survey, only 10% of 744 levees from National Levee Database were rated as acceptable, while the rest were marginally acceptable or unacceptable.1 Formidable failure of levees in New Orleans during hurricane Katrina along the Mississippi River resulted in great loss economically with human casualties in that area. This vivid catastrophe highlighted the importance of levee monitoring. Extensive research has been done to monitor levee status with various different approaches. The main approaches used are field-based in situ soil property measurements and remote sensing measurements [e.g., synthetic aperture radar (SAR) images, optical images]. Recently, research on screening levees has been conducted at Mississippi State University.2–5 Airborne and spaceborne SAR images are used to monitor the abnormality of study areas along the Mississippi River. Another similar application is landslide monitoring, where SAR images may be combined with optical images and digital terrain models.6,7 In this research, an airborne SAR having three magnitude bands with polarizations HH, VV, and HV was used for levee monitoring, and supervised classification of slide and nonslide levee areas was performed using the standard support vector machine (SVM) classifier.8–11 Although SVM is a powerful classifier, it failed to accurately classify the slide areas without any spatial feature extraction. This may be because the original low-dimensional data does not include enough discriminant features. Thus, unlike a dimensionality reduction process for high-dimensional data,12–14 a dimensionality expansion process is included by adding additional artificial bands6,15 which are simply nonlinearly normalized bands in this research. In addition, a feature extraction method for spatial information has been considered before classification. Gray-level co-occurrence matrix (GLCM) is one of the popular methods to describe spatial features, and it performs well in many applications.16–18 It has been applied to levee slide classification.5,19 However, it requires significant computational power, since a relatively large local area has to be considered in order to find accurate texture features. Furthermore, several position operators have to be applied and statistical features must be calculated from the GLCM before classification. Thus, it may not be suitable for fast analysis of a big image.
In this study, very simple and effective feature extraction methods for SAR images are proposed. Specifically, the weighted average filter and the majority filter20 may offer slightly lower classification accuracy but with much less computational cost. Such spatial low-pass filtering techniques with a sliding window are suitable for parallel computing, since the output of a single pixel is unrelated to other areas of the image.21 Thus, such spatial feature extraction methods are preferred for fast processing of large-scale remote sensing images. When the GLCM is used, adding the features from such spatial filters may further improve the performance. For complicated slide features, the use of additional normalized bands improves the performance of GLCM and spatial filtering.
The contributions of this study are threefold:
1. Spatial low-pass filters are proposed for spatial feature extraction in levee slide classification from SAR images. The center pixel is replaced by the filter output of a small neighborhood, which includes spatial information; meanwhile, the spatial low-pass filters can reduce noise for classification performance enhancement. It can also be easily implemented in parallel for fast data processing and analysis.
2. For GLCM, adding the low-pass filtering outputs may further improve classification accuracy.
3. We propose to use normalized bands which can generate additional spatial and texture features to improve the classification accuracy.
This paper is organized as follows. Section 2 introduces the data used in the experiments. Section 3 presents the methods proposed in this paper. Section 4 shows experimental results. The conclusions are drawn in Sec. 5.
As shown in Fig. 1, the study area in this paper is an ~3-km-long portion of the levee system on the eastside of the Mississippi River, north of Vicksburg, Mississippi. About a 40-m-wide mask (buffer) on the river side from the levee road was applied to segment the area for classification, as illustrated in Fig. 2.
The imagery for the study was taken by the NASA JPL’s uninhabited aerial vehicle synthetic aperture radar (UAVSAR) on June 16, 2009. UAVSAR acquires repeat-track SAR data to get differential interferometric measurements.22 Reconfigurable polarimetric L-band SAR sensors are mounted. The bandwidth of the radar is 80 MHz and it has 1.8 m resolution with full polarimetry. Its noise level is relatively low. Thus, it has the capability to differentiate targets with weak radar backscattering cross-section. L-band SAR can penetrate a few meters in very dry soil, but its penetration is typically a few centimeters. Therefore, the imagery from UAVSAR is an excellent source for monitoring levee change.2,23
Figure 2 shows levee slide locations in two areas of interest (AOIs) used in the experiments: AOI1 is a image and AOI2 is an image. AOI1 has more complicated texture features than AOI2. All the pixels are labeled. Different percentages of labeled samples are used as training samples and the rest for testing in the experiments.
The strength of radar backscatter is heavily affected by surface roughness of the terrain and a slide is identified as rough patch.23 Features responsive to surface roughness include the magnitudes of the HH, VV, and HV polarimetric backscattering coefficients. Sometimes, classification can be done successfully on the original images without spatial feature extraction. But in the case of using SAR images for levee slide classification, it is almost impossible to achieve accurate classification without proper spatial feature extraction.
There are many approaches to extract spatial features. GLCM is commonly used to extract texture features. GLCM is a versatile method with many choices for operators and features. On the contrary, it has many parameters to tune for a specific application, such as window size, level of gray scale, direction and distance from a center pixel, etc. A common approach for GLCM is to choose as many features as possible, followed by a feature dimensionality reduction method. In this study, since processing efficiency is a concern, very simple feature extraction techniques, i.e., the weighted average filter and the majority filter,20 are adopted to capture spatial property as an alternative to GLCM. It is well known that GLCM is prone to be affected by noise. For noisy SAR images, spatial low-pass filters can reduce noise. Moreover, the spatial low-pass filters require much lower computational cost, and they can be easily implemented in parallel.
GLCM and Band Normalization
A GLCM relates a pixel to other pixels with specific distance and direction defined by a position operator. The position operator is application-specific. Sometimes, several different operators have to be used together. After a GLCM is generated, several quantitative features, such as homogeneity, uniformity, contrast, and entropy have to be computed before the related texture information can actually be used for classification. Therefore, it may be complicated for the GLCM technique to be applied for large-scale images in remote sensing, and it is difficult to be implemented in parallel. There are dozens of quantitative textural features that can be derived from GLCM, but only four major features, as shown in Table 1, are used in this paper since most of the others are either insensitive to levee slides or are highly correlated to these four.
The major features of gray-level co-occurrence matrix (GLCM) used in this research.
The magnitude of the SAR image varies within a certain range based on levee condition. This could be problematic with setting a proper level of gray scale for GLCM generation. Therefore, a fractional measure for relative backscatter strength among HH, VV, and HV bands is derived as
Then the values in the three bands after normalization are fractional within [0 1]. To be applicable to GLCM generation, all the values are converted to integers (after multiplying by 100). Note that such a normalization process does not actually change the polarization feature in a pixel vector, but it does change the neighboring spatial feature of the pixel in each band because the normalization term in Eq. (1) is varied per pixel. In the experiments, it will be demonstrated that using normalization bands can improve the performance of GLCM-based feature extraction.
Spatial filtering is widely used in digital image processing. A small square window is often employed to slide over an entire image; the filter output at each location is assigned to the center pixel of the window. For multidimensional image processing, the filter output includes both spatial and spectral information. A low-pass spatial filter, such as a local averaging filter, can reduce noise but smooth out image details such as edges.
A weighted average filter, which actually is the Gaussian low-pass filter, is often employed. Its weight in a local window is defined by
Another option is to apply a majority filter. The image needs to be quantized to integer levels similar to GLCM. Proper window size and quantization level should be decided for a specific application. In the case of each window, for example, the majority filter assigns the predominant value to the center pixel. If no predominant value is found, or when all the nine input pixels have different values, then the median value in the window is used for the center pixel. Compared to the weighted average filter, the majority filter can better maintain image details.
A few combinations of feature sets are investigated for comparison purposes, which are described below (with the number of features).
1. OR: original three bands (3)
2. ON: original three bands + normalized bands ()
3. OA: original three bands + weighted average filter ()
4. OM: original three bands + majority filter ()
5. OAM: original three bands + weighted average filter + majority filter ()
6. ONM: original three bands + normalized bands + majority filter ()
7. ONAM: original three bands + normalized bands + weighted average filter + majority filter ()
8. OG: original three bands + GLCM ()
9. ONG: original three bands + normalized bands + GLCM ()
10. OGM: original three bands + GLCM + majority filter ()
11. ONGM: original three bands + normalized bands + GLCM + majority filter ()
These combinations (including the original bands, nonlinearly generated bands, and extracted spatial features) present discriminant quantities in different domains (polarizations, their correlations, and spatial information) for better classification performance.24 For each combination, the SVM with a radial basis function kernel and cross-validation-tuned parameters was applied; 10 to 50% randomly selected pixels were used as training samples and the remaining for testing. A total of 20 runs were made and the average performance was reported. Table 2 summarizes the parameter settings for spatial feature extraction.
Parameter settings for both GLCM and spatial filtering based feature extraction.
|Weighted average filter||Majority filter||GLCM|
|σ||Window size||Level||Window size||Level||Window size||Distance||Direction|
|AOI1||5||5||9||7||9||7||1||0 deg, 45 deg, 90 deg, 135 deg|
|AOI2||5||3||9||7||9||7||1||0 deg, 45 deg, 90 deg, 135 deg|
Figure 3 shows the classification results for AOI1 and AOI2 with different percentages of labeled samples. Table 3 further summarizes the performance when using 30% of the labeled samples. The classification performance was evaluated in terms of overall accuracy. As shown in Table 3, SVM could not well classify slide pixels directly from the original three bands (denoted as OR). Applying an average filter or a majority filter to the original bands (i.e., OA and OM), respectively, slightly improves the performance, while applying both of these low-pass filters (i.e., OAM) improves the performance further. After adding the normalized bands (i.e., ONM, ONAM), the classification accuracy is increased (compared to the counterparts OM and OAM, respectively). Once GLCM is deployed to generate texture features, the results are enhanced for AOI1, which has more complicated texture features than AOI2; it seems that applying GLCM to the original bands (i.e., OG) works well for the simple texture in AOI2. However, when the majority filter and weighted average filter are applied to the original bands and normalized bands (i.e., ONAM), the results are comparable to the results from GLCM (i.e., OG). For AOI1, ONAM has overall accuracy of 0.9312, while OG yields 0.9305; the overall accuracy is 0.9820 for ONAM and 0.9832 for OG in the case of AOI2. Obviously, the difference is marginal. Figure 4 shows classification maps of OAM, ONAM, and ONGM of the two study areas. Compared to the ground truth maps, those from ONAM look quite similar to those from ONGM, although the classification accuracy values of ONAM in Table 3 are slightly lower than those of ONGM.
Classification performance when using 30% for training (and 70% for testing).
When normalized bands are added, classification accuracy can be improved in general. For instance, ONM is better than OM, and ONAM is better than OAM. The improvement from using normalized bands together with GLCM is also obvious (ONG is better than or similar to OG, and ONGM is better than or similar to OGM). Since the normalization process changes the neighboring spatial feature of a pixel in each band, a spatial filter or a GLCM on these bands provides additional spatial or texture features, thereby yielding performance enhancement.
Table 4 shows execution time (and feature dimensionality) for the representative combinations in Table 3. The experiment was done on a computer with Intel Xeon CPU (3.20 GHz) and 6G memory. While the results from ONAM and OG are comparable, the execution time has a very big gap for these two small images. ONAM method takes only 3.81 s to classify the AOI1 image and 18.86 s for OG. If GLCM is applied to the one with a larger feature dimensionality as in ONGM, computing time increases exponentially to 58.16 s. To classify the AOI2 image, it costs ONAM 7.08 s, OG 37.75 s, and ONGM 121.39 s. The same tendency could be found for those using GLCM features: if GLCM has to be applied to a large-scale image, it will take a much longer time; however, the weighted average filter and the majority filter can extract spatial features with much less computing time. Therefore, the ONAM method can be a very promising and efficient approach to handle fairly large images, such as spaceborne images, with less computational complexity.
Execution time in seconds.
|Study area||AOI1 (66×48×3)||AOI2 (80×83×3)|
|Method||Execution time||Feature dimension||Execution time||Feature dimension|
Discussion and Conclusion
Efficient spatial feature extraction approaches are investigated for levee slide classification using SAR images. GLCM feature extraction performs well as expected but has limitations of high computation cost and storage needs, and the entire process is difficult to implement in parallel. For small datasets, this may not be crucial; but as data size increases, efficient feature extraction is needed. Spaceborne and airborne image applications usually involve large images. In this paper, an average filter and a majority filter were shown to have much lower computational cost and can be easily implemented in parallel to further reduce computing time. Even though their classification performance does not exceed GLCM (in some feature combinations), they can be used for fast screening before more complicated methods are applied on a small selected area. The spatial features and feature combinations investigated in this research may be useful for early abnormality screening with a technical of anomaly detection.25–27
In high-dimensional data analysis (e.g., hyperspectral image classification), generating spatial features will dramatically increase the feature dimensionality and aggravate the problem of the “curse-of-dimensionality.” For a polarized SAR image, this is not a problem; actually, lack of sufficient discriminant features in a low-dimensional SAR image is the difficulty. The nonlinear band generation process intends to dig out the information embedded in the original data that can be used to maximize class separability, which in this research is about band normalization, although many nonlinear combinations can be tested.15 Such a normalization step is used to generate a new three-dimensional unit vector, so the polarization vector shape (instead of magnitude) can be emphasized; meanwhile, each pixel is divided by a different value, creating three different polarization bands with new spatial information for discrimination.
For future work, we will study parallel implementation of the proposed methods and anomaly detection for screening of potential levee slides.
This work is partially supported by the National Science Foundation under Award No. OISE-1243539 and by the NASA Applied Sciences Division under Grant Number NNX09AV25G. U.S. Army Corps of Engineers, Engineer Research and Development Center, and Vicksburg Levee District provided ground truth data.
Deok Han is currently a PhD student in the Department of Electrical and Computer Engineering of Mississippi State University and works as a graduate research assistant in the Geosystems Research Institute (GRI) at Mississippi State University. He holds a bachelor’s degree in industrial automation engineering from Inha University, Korea (1997), and master’s degrees in electrical and computer engineering from Mississippi State University (2004). His research interests include machine learning, signal/image processing, hyperspectral remote sensing image, and synthetic aperture radar.
Qian Du received her PhD degree in electrical engineering from the University of Maryland Baltimore County in 2000. Currently, she is Bobby Shackouls professor in the Department of Electrical and Computer Engineering at Mississippi State University. Her research interests include hyperspectral remote sensing image analysis, pattern classification, data compression, and neural networks. She is an associate editor for Journal of Applied Remote Sensing. She is a Fellow of SPIE.
James V. Aanstoos is an associate research professor in the Geosystems Research Institute (GRI) at Mississippi State University and also a senior lecturer in the Geological and Atmospheric Sciences Department at Iowa State University. He holds bachelor’s and master’s degrees in electrical engineering from Rice University (1977, 1979) and a PhD in atmospheric science from Purdue (1996). His research interests include remote sensing, synthetic aperture radar, and machine learning.
Nicolas H. Younan is currently the department head and James Worth Bagley Chair of Electrical and Computer Engineering at Mississippi State University. He received BS and MS degrees from Mississippi State University in 1982 and 1984, respectively, and the PhD degree from Ohio University in 1988. His research interests include signal processing and pattern recognition. He has been involved in the development of advanced signal processing and pattern recognition algorithms for data mining, data fusion, feature extraction and classification, and automatic target recognition/identification.